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EXECUTIVE  SUMMARY 


The  Ada  community,  including  government,  industry,  and  academic  personnel, 
needs  the  capability  to  assess  APSEs  (Ada  Programming  Support  Environments)  and  their  com¬ 
ponents,  and  to  determine  their  conformance  to  applicable  standards  (e.g.,  DoD-STD-1838,  the 
CAIS  standard).  The  technology  required  to  fully  satisfy  this  need  is  extensive  and  largely  un¬ 
available;  it  cannot  be  acquired  by  a  single  government-sponsored,  professional  society- 
sponsored,  or  private  effort.  The  purpose  of  the  APSE  Evaluation  and  Validation  (E&V)  Task  is 
to  provide  a  focal  point  for  addressing  the  need  by: 

<v£l)  Identifying  and  defining  specific  technology  requirements, 

:  (2)  Developing  selected  elements  of  this  technology. 

(3)  Encouraging  others  to  develop  additional  elements;  and 
o  (4)  Collecting  information  describing  elements  which  already  exist. 

This  information  will  be  made  available  to  DoD  components,  other  government  agencies,  indus¬ 
try  and  academia. 

The  purpose  of  the  E&V  Reference  Manual  (this  document)  is  to  provide  information 
that  will  help  users  to: 

Hi)  Gain  an  overall  understanding  of  APSEs  and  approaches  to  their 
assessment; 

c*-(2)  Find  useful  reference  information  (e.g.,  definitions)  about  specific 
elements  and  relationships  between  elements;  and 

:  (3)  Find  criteria  and  metrics  for  assessing  tools  and  APSEs,  and 
techniques  for  performing  such  assessments.  r 

The  last  are  to  be  found  (or  referenced)  in  a  companion  document  called  the  E&V  Guidebook. 

Chapters  1  through  3  provide  a  general  introduction  to  the  document  and  to  the  is¬ 
sue  of  assessing  APSEs  as  a  whole. 
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Chapter  4  and  later  chapters  are  “formal  chapters”  built  around  a  standard  format 
and  formal  grammar.  Each  of  the  formal  chapters  corresponds  to  one  index  of  an  overall  E&V 
Classification  Schema.  The  schema  adopts  a  relational  model  of  the  subject  and  process  of 
E&V.  This  model  will  allow  the  user  to  arrive  at  E&V  techniques  through  many  different  paths, 
and  provides  a  means  to  extract  useful  information  along  the  way. 

Yearly  updates  and  extensions  to  this  manual  are  planned.  Therefore,  comments 
and  suggestions  are  welcome.  Please  send  comments  electronically  (preferred)  to 
szymansk@ajpo.sei.cmu.edu  or  by  regular  mail  to  Mr.  Raymond  Szymanski,  WRDC/AAAF, 
Wright  Patterson  AFB,  OH  45433-6543. 
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1. 


INTRODUCTION 


1.1  PURPOSE  OF  THE  MANUAL 

This  document  is  a  product  of  the  Ada  Programming  Support  Environment  (APSE) 
Evaluation  and  Validation  (E&V)  Task  sponsored  by  the  Ada  Joint  Program  Office.  It  is  one  of  a 
pair  of  companion  documents  known  as  the  E&V  Reference  System,  consisting  of: 

•  E&V  Reference  Manual 

•  E&V  Guidebook. 


The  purpose  of  the  Reference  Manual  is  to  provide  a  collection  of  information  to  support  a  variety 
of  users.  The  collection  is  organized  in  accordance  with  a  Classification  Schema  described  in 
Chapter  2.  It  should  help  users  to: 


•  Gain  an  overall  understanding  of  APSEs  and  approaches  to  the 
assessment  of  APSE  performance,  quality  and  conformance  to 
applicable  standards. 

•  Find  useful  reference  information,  such  as  definitions  of  specific 
elements  of  the  Classification  Schema,  and  relationships  be¬ 
tween  elements. 

•  Find  criteria  and  metrics  for  assessing  specific  components, 
combinations  of  components  and  "whole  APSEs,”  and  locate 
relevant  E&V  techniques. 


The  Reference  Manual  includes  many  "pointers”  to  sections  in  the  Guidebook  and  other  docu¬ 
ments  which  describe  E&V  techniques.  Figure  1.1-1  illustrates  the  relationship  between  the  Ref¬ 
erence  Manual  and  the  Guidebook.  Figures  1.1-2  and  1.1-3  illustrate  the  types  of  information  to 
be  extracted  from  each  document.  Chapter  2  provides  a  more  detailed  description  of  the  struc¬ 
ture  and  uses  of  the  Reference  System. 
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Directly  from 
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E&V  Tools  and  Techniques 


Figure  1.1-1  Uses  of  the  Reference  System 
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Users  consult  Indexes  in  the  Reference  Manual 


J  V 
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EXAMPLES  of  Types  of  Information  Extracted: 

•  Organization  of  The  Indexes 

Hierarchical  Structure  and  a  Numbering  System 
(e.g.,  The  Function  Index  .  .  .  see  Fig.  2.1-2) 

•  Names  and  Descriptions  of  Specific  Elements  within  the  Indexes 

(e.g.,  The  Compilation  Function  .  .  .  see  Figs.  2.1-2  and  2.1-3) 

•  Cross  References  Between  Elements  of  the  Indexes 

(e.g.,  The  Function  “Compilation”  is  Cross  Referenced  to 
Activities  and  Tools  .  .  .  see  Fig.  2.4-1) 


Figure  1.1-2  Examples  of  Reference  Manual  Information 


Pointers 


G-00926a 

8-14-89 


E&V 

Guidebook 


EXAMPLES  of  Types  of  Information  Extracted: 

•  Description  of  Evaluation  (E)  Tools  and  Their  Uses  (e.g..  see  Section  2.4) 

•  Description  of  Validation  (V)  Suites 

•  References  to  Other  Documents  Describing  E  or  V  Techniques 

•  Synopses  of  Other  Documents  That  Cover  Multiple  Techniques 


Figure  1.1-3  Examples  of  Guidebook  Information 
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1.2  USERS  OF  THE  MANUAL 

Classes  of  people  who  are  expected  to  be  users  of  this  manual  are  listed  in  Ta¬ 
ble  1.2-1.  They  are  described  in  terms  of  their  relationships  to  deliverable  software,  tools, 
APSEs,  and  APSE  E&V  technology.  They  may  be  associated  with  Government,  industry,  or  aca¬ 
demia.  The  table  was  adapted  from  material  in  the  report  of  the  E&V  Workshop  [@  E&V  Work¬ 
shop  1984].* 

While  the  manual  is  designed  to  be  of  use  to  people  of  all  the  classes  listed,  the  G£i- 
mary  users  are  expected  to  be  the  APSE/tool  users  and  the  E&V  technology  users.  These  are 
people  with  highly  technical  backgrounds  and  technical/managerial  interests  in  evaluating  and 
selecting  tools  and  APSEs.  This  expectation  has  strongly  influenced  the  structure  and  style  of 
the  manual.  The  primary  users  are  likely  to  consult  both  the  Reference  Manual  and  the  Guide¬ 
book.  Many  of  the  other  classes  of  users  listed  in  Table  1 .2-1  are  likely  to  consult  the  Reference 
Manual  only. 


Table  1.2-1  Reference  Manual  Users 


CLASS 

DESCRIPTION 

Software  Acquisition 
Personnel 

Government  program  officers  and  commercial 
program  managers  who  let  contracts  for 
software  development 

APSE/Tool  Users 

Project  managers,  librarians,  system 
engineers,  software  engineers 

APSE/Tool  Builders 

Environment/tool  designers,  implementers, 
managers,  marketing  personnel 

E&V  Technology  Users 

Government,  commercial  and  university 
personnel  applying  E&V  technology 

E&V  Technology 

Builders 

Anyone  developing  E&V  assessment  techniques 
(E&V  Task  contractors,  etc) 

Investors 

Anyone  funding  development  or  use  of  E&V 
technology  (Congress,  AJPO,  corporations,  etc.) 

*The  format  used  for  references  is  associated  with  the  "formal  grammar”  used 
beginning  with  Chapter  4.  See  further  explanations  in  Appendix  C. 


1-4 


E&V  Reference  Manual,  Version  2.0 


1.3  THE  NEED  FOR  E&V  TECHNOLOGY 

Technology  for  the  assessment  of  APSEs  and  APSE  components  (tools)  is  needed 
because  of  the  difficulty  in  assessing  APSEs  and  because  of  the  importance  of  the  decisions 
made  based  on  these  assessments.  The  importance  of  an  APSE  selection  is  evident  when  one 
considers  the  large,  critical,  Ada-based  systems  to  be  developed  in  the  coming  years.  The  effec¬ 
tiveness,  reliability,  and  cost  of  these  systems  will  be  strongly  influenced  by  the  environments 
used  to  develop  and  maintain  them.  From  the  point  of  view  of  a  software  developing  organization, 
the  decision  to  select  an  APSE  can  be  an  important  investment  decision  with  long-lasting  influ¬ 
ence  on  a  number  of  projects  and  the  organization’s  methods  of  operation,  training,  and  competi¬ 
tiveness.  From  the  point  of  view  of  a  software  maintenance  organization,  the  environment  used 
will  strongly  influence  the  organization’s  effectiveness,  as  well  as  the  cost  of  its  operations  and 
training. 


The  difficulty  of  assessing  APSEs  and  tools  exists  for  several  reasons.  First,  an 
APSE  represents  very  complex  technology  with  many  elements,  which  can  be  assessed  individu¬ 
ally  or  in  combination.  Second  there  is  a  confusing  diversity  of  choice  with  respect  to  individual 
tools,  tool  sets,  or  “whole  APSEs”;  and  there  are  a  number  of  ways  of  viewing  APSEs;  see  Chap¬ 
ter  3.  Third,  the  state  of  the  art  of  APSE  architecture  and  of  some  categories  of  tools  (e.g., 
graphic  design  tools)  is  undergoing  rapid  change.  Finally,  there  is  a  lack  of  historical  data  rele¬ 
vant  to  APSEs,  partly  because  of  the  general  pace  of  technological  change  and  partly  because 
we  are  dealing  with  the  Ada  language,  a  relatively  new  implementation  language.  E&V  technology 
provides  methods  and  techniques  to  overcome  these  difficulties  and  provides  a  basis  for  deter¬ 
mining  performance  and  other  attributes  of  APSEs. 

In  addition  to  the  need  for  assessment  technology  itself,  there  is  a  need  for  informa¬ 
tion  about  this  technology.  Potential  buyers  and  users  of  APSEs  and  tools  need  a  framework  for 
understanding  APSEs  and  their  assessment,  as  well  as  information  about  specific  assessment 
techniques.  Similarly,  vendors  of  tools  and  APSEs  need  to  be  aware  of  the  deficiencies  of  current 
products,  as  well  as  the  criteria  to  be  used  in  the  assessment  of  future  products.  Such  aware¬ 
ness  on  both  sides,  expressed  in  a  common  terminology,  should  speed  up  the  evolution  of  better 
software  development  environments. 
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1.4  BACKGROUND 

In  June  1983  the  Ada  Joint  Program  Office  (AJPO)  proposed  the  formation  of  the 
E&V  Task  and  a  tri-service  E&V  Team,  with  the  Air  Force  designated  as  lead  service.  In  October 
1983  the  Air  Force  officially  accepted  responsibility  as  lead  service  and  designated  the  Air  Force 
Wright  Aeronautical  Laboratories  (AFWAL)  at  Wright  Patterson  Air  Force  Base  as  lead  organiza¬ 
tion.  In  April  1984  an  E&V  Workshop  was  held  at  Airlie,  Virginia.  The  purpose  of  the  workshop 
was  to  solicit  participation  of  industry  representatives  in  the  E&V  Task.  Many  of  the  participants 
in  the  workshop  have  chosen  to  remain  involved  as  Distinguished  Reviewers,  and  additional  in¬ 
dustry  participants  have  subsequently  become  involved  in  E&V  Team  activities. 


The  E&V  Team  publishes  an  annual  public  report.  The  following  paragraph  is 
quoted  from  the  1987  version  [@E&V  Report  1987]  of  the  report: 


“The  Ada  community,  including  government,  industry,  and  academic  personnel, 
needs  the  capability  to  assess  APSEs  (Ada  Programming  Support  Environ¬ 
ments)  and  components,  and  to  determine  their  conformance  to  applicable 
standards  (e.g..  DoD-STD-1838,  the  CAIS  standard).  The  technology  required 
to  fully  satisfy  this  need  is  extensive  and  largely  unavailable:  it  cannot  be  ac¬ 
quired  by  a  single  government-sponsored,  professional  society-sponsored,  or 
private  effort.  The  purpose  of  the  APSE  Evaluation  and  Validation  (E&V)  task 
is  to  provide  a  focal  point  for  addressing  the  need  by  (1)  identifying  and  defin¬ 
ing  specific  technology  requirements,  (2)  developing  selected  elements  of  the 
required  technology,  (3)  encouraging  others  to  develop  some  elements,  and  (4) 
collecting  information  describing  existing  elements.  This  information  will  be 
made  available  to  DoD  components,  other  government  agencies,  industry,  and 
academia.” 


The  team  public  reports  contain  much  additional  information  for  the  interested 
reader.  Three  competitive  contracts  have  been  awarded  under  the  E&V  task.  These  are: 


•  Technical  Support  contract  -  awarded  June  1985 

•  Ada  Compiler  Evaluation  Capability  (ACEC)  contract  -  awarded 
February  1987 

•  CAIS  Implementation  Validation  Capability  (CIVC)  contract  - 
awarded  May  1987. 
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The  major  purpose  of  the  first  of  these  contracts  is  to  create  and  update  elements  of  the  E&V 
Reference  System,  including  this  manual.  The  purpose  of  the  second  and  third  contracts  is  to 
create  two  specific  elements  (ACEC  and  CIVC)  of  the  needed  E&V  technology. 


1.5  ORGANIZATION  OF  THE  MANUAL 

Chapter  2  discusses  the  structure  of  the  E&V  Reference  System  (Reference  Manual 
plus  Guidebook)  and  the  Classification  Schema  upon  which  that  structure  is  based.  Specific  di¬ 
rections  as  to  how  to  use  the  manual  are  also  included. 

Chapter  3  provides  a  general  discussion  of  “whole  APSE”  issues,  in  which  an  APSE 
is  viewed  as  more  than  the  sum  of  its  parts.  Key  whole-APSE  attributes  are  discussed  along  with 
general  approaches  to  whole-APSE  assessment. 

Chapter  4  and  subsequent  chanters  are  “formal  chapters”  built  around  a  standard 
format  and  formal  grammar.  Each  of  the  formal  chapters  corresponds  to  one  index  of  the  Classi¬ 
fication  Schema.  The  structure  and  use  of  these  chapters  are  the  focus  of  the  material  found  in 
Chapter  2. 


The  appendices  include  a  description  of  the  formal  grammar,  a  glossary  of  acro¬ 
nyms  and  abbreviations,  a  document  citation  list,  and  a  composite  index. 
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2.  USE  OF  THE  REFERENCE  SYSTEM 


This  chapter  provides  a  step-by-step  explanation  of  how  to  use  the  E&V  Reference 
System  and  the  Classification  Schema  upon  which  the  system  is  based.  Section  2.1  describes 
the  organization  of  the  material.  Sections  2.2,  2.3,  and  2.4  contain  illustrations  of  uses  of  the 
system,  presented  in  increasing  levels  of  sophistication.  Section  2.5  provides  a  global,  concep¬ 
tual  view  of  the  system  framework.  User  A  (Section  2.2)  consults  an  index  of  the  Reference 
Manual  to  find  the  description  of  a  term  that  is  an  element  of  that  index.  User  B  (Section  2.3) 
consults  an  index  to  find  an  element  and  several  cross  references  to  related  elements  in  another 
index.  User  C  (Section  2.4)  consults  a  combination  of  indexes  to  find  references  to  sections  in 
the  Guidebook,  which  contain  explanations  of  relevant  Evaluation  or  Validation  techniques.  Brief 
definitions  of  several  key  words  and  expressions  follow: 

E&V  —  Evaluation  and  Validation 

Evaluation  —  Assessing  performance  and  quality 

Validation  —  Assessing  conformance  to  a  standard 

E&V  Reference  System  —  Two  documents: 

the  E&V  Reference  Manual  and 
the  E&V  Guidebook 

E&V  Classification  Schema  —  A  set  of  indexes  that  provide  a 

framework  for  the  E&V  Reference 
Manual. 

The  schema  was  initially  described  in  an  earlier  “E&V  Classification  Schema  Report"  [@E&V 
Classification].  Subsequent  changes  in  the  schema  will  be  updated  in  future  versions  of  this 
manual:  the  schema  report  will  not  be  updated. 

2.1  SYSTEM  ORGANIZATION 

The  entire  reference  system  can  be  viewed  as  a  structure  of  multiple  indexes.  For 
example,  there  is  a  function  index  and  a  life  cycle  activity  index,  among  others.  The  structure  is 
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analogous  to  the  card  catalog  system  in  a  public  library,  with  its  author  index,  title  index,  and  sub¬ 
ject  index.  To  use  the  card  catalog  in  the  library,  you  must  first  locate  the  card  that  corresponds 
to  the  author,  title,  or  subject  in  which  you  are  interested.  Similarly,  to  use  the  Reference  Manual, 
you  first  find  the  element(s)  in  which  you  are  interested.  The  way  to  do  this  is  to  begin  by  looking 
at  the  Table  of  Contents  or  the  Index  at  the  back  of  the  manual  to  locate  the  element (s)  in  the 
“formal  chapters.”  The  names  of  the  indexes  that  are  formal  chapters  within  the  Reference 
Manual  are: 


•  Life  Cycle  Activities  (Chapter  4) 

•  APSE  Tool  Categories  (Chapter  5) 

•  Attributes  (Chapter  6) 

•  Functions  (Chapter  7). 

Figures  2.1-1, 2.1-2,  and  2.1-3  provide  a  pictorial  view  of  the  organization  of  the  Ref¬ 
erence  Manual,  particularly  the  structure  of  the  reference  material  contained  in  the  “middle  sec¬ 
tion.  "  The  chapters  of  this  middle  section  are  organized  in  a  consistent,  formal  manner,  using  a 
formal  grammar  (described  in  Appendix  C).  A  typical  chapter  corresponds  to  one  index  of  the 
schema.  A  typical  index  is  organized  as  a  hierarchical  structure  of  elements.  For  every  element 
there  is  a  “text  frame”  that  has  (in  general)  three  parts  as  shown  in  Fig.  2.1-3.  The  text  frames 
are  built  using  the  formal  grammar.  (Details  of  the  formal  grammar  need  not  concern  the  user.  It 
was  employed  because  of  the  possibility  of  a  future  on-line,  electronic  version  of  the  system, 
supported  by  advanced  updating  and  information  retrieval  techniques.) 
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The  E&V  Reference  Manual 
Has  Three  Parts,  As  Shown 


G-00927b 


Appendices  &  Index 


,  *  T'T;  i  Reference 

Introductory  Materja| 

Chapters 

Typically  a  chapter  in  the  "Reference  Material"  section  corresponds,  to  one 
index  of  the  Classification  Schema.  These  chapters  are  organized  in  a  consistent, 
formal  manner. 


Figure  2.1-1  Reference  Manual  Organization 
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Description:  Is  a  brief  definition  of  the  element. 

Cross  References:  Identify  significant  relationships  with  other  elements 

in  the  same  index  or  other  indexes  of  the  Reference 
Manual. 

Guidebook  References:  Identify  relationships  involved  in  the  statements  of 

E&V  objectives  and  point  to  sections  in  the  Guide¬ 
book  where  information  about  how  to  satisfy 
objectives  may  be  found. 


Figure  2.1-3  Text  Frames 
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2.2  DESCRIPTION:  DIRECT  REFERENCE 

Our  first  example  of  the  use  of  the  Reference  Manual  is  “User  A"  who  consults  the 
Attributes  Index  to  find  the  description  of  the  term  “Storage  Effectiveness.”  Figure  2.2-1  is  a 
(truncated)  copy  of  Section  (or  Text  Frame)  6.4.31 .  User  A  may  find  this  frame,  for  example,  by 
browsing  through  the  Table  of  Contents  or  by  looking  up  the  term  “Storage  Effectiveness”  in  the 
main  Index  at  the  back  of  the  manual.  Note  that  the  text  frame  contains  Cross  References  and 
Guidebook  References  as  well,  but  this  need  not  concern  User  A,  who  simply  seeks  a  descrip¬ 
tion.  The  User  A  scenario  is  pictorially  represented  in  Fig.  2.2-2,  where  the  boxes  are  analogous 
to  cabinets  in  a  library  card  catalog  system. 


6.4.31  Storage  Effectiveness 
Description: 

Those  characteristics  of  the  software  which  provide  for  minimum  utilization  of  storage  re¬ 
sources  in  performing  functions.  Metrics  include:  the  use  of  virtual  storage,  dynamic  realloca¬ 
tion  of  memory,  and  the  avoidance  of  redundant  storage  of  data.  [@RADC  1985]  The  choice 
between  alternative  source  code  constructions  based  on  those  taking  the  minimum  number  of 
words  of  object  code  or  in  which  the  information-packing  ...  is  high.  [@DACS  1979] 

Cross  References; 


Software  Quality  Factors: 


[Efficiency 

6.1.1] 

Beneficial  Quality  Factors: 

Adverse  Quality  Factors: 

[Maintainability 

6.2.2, 

Verifiability,  Testability 

6.2.3, 

Transportability 

6.3.4] 

Guidebook  References: 

[@GB:  RADC  Software  Quality  Metric  Worksheets 

99.5; 

@GB:  SEI  Assessment  of  Software  Engineering  Tools 

99.6: 

Preliminary  Design 

7. 1.6. 4, 

@GB:  SEI  Design  Support  Experiment 

9.1; 

Detailed  Design 

7. 1.6. 5, 

@GB:  SEI  Design  Support  Experiment 

9.1; 

'Compilation 

7.1. 6.7, 

@GB:  IDA  Benchmarks 

5.2; 

'Compilation 

7.1. 6. 7, 

@GB:  ACEC 

5.3; 

'Compilation 

7. 1.6. 7, 

@GB:  PIWG  Benchmark  Tests 

5.4; 
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2.3  CROSS  REFERENCE 

Our  second  example  of  use  of  the  Reference  Manual  is  "User  B”  who  would  like  to 
know  the  names  and  descriptions  of  functions  associated  with  a  particular  group  of  life  cycle  ac¬ 
tivities.  User  B  first  consults  the  Life  Cycle  Activities  Index,  Chapter  4,  which  divides  the  life  cy¬ 
cle  in  accordance  with  the  DoD-STD-2167A  model  [@DoD-STD-2167A]  for  project  development. 
Chapter  4  relates  the  life  cycle  activity  groups  to  the  functions  typically  found  within  an  APSE  and 
the  products  that  are  produced  in  each  portion  of  the  life  cycle.  A  sample  Life  Cycle  Activities  In¬ 
dex  Frame  is  shown  in  Fig.  2.3-1 .  This  frame  represents  the  sixth  activity  group  [4.6  Coding  and 
CSU  Testing]  and  the  second  class  of  activities  [4.6.2  Software  Engineering]  covered  in  each 
activity  group.  The  Software  Development  File  is  the  product  for  this  group  of  activities.  User  B 
finds  61  functions  that  might  be  performed  during  this  activity  listed  in  the  text  frame.  In  each 
case  another  text  frame  is  cross  referenced  in  Chapter  7,  the  Functions  Index,  where  more  infor¬ 
mation  about  this  function  is  found.  The  User  B  scenario  is  pictorially  represented  in  Fig.  2.3-2. 
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4.6.2  Software  Engineering 


Cross  References: 

Products: 

[Software  Development  File 

Functions: 

[Data  Editing 

Predefined  and  User-Defined  Forms 

Assembling 

Compilation 

Conversion 

Macro  Expansion 

Structure  Preprocessing 

Body  Stub  Generation 

Preamble  Generation 

Linking/Loading 


(SDF)] 

7.1. 1.2, 

7.1. 2. 3, 

7. 1.6. 6, 

7. 1.6. 7, 

7. 1.6. 8, 

7. 1.6. 9, 

7.1.6.10, 

7.1.6.11, 

7.1.6.12, 

7.1.6.13, 


Mutation  Analysis 
Symbolic  Execution 
Regression  Testing 
Emulation 
Tuning 

Reliability  Analysis 
Real  Time  Analysis 
Path  and  Domain  Selection 
Formal  Verification 


7. 3. 2. 9, 

7.3.2.10, 

7.3.2.11, 
7.3.2.13, 

7.3.2.15, 

7.3.2.16, 

7.3.2.17, 

7.3.2.18, 
7.3.3] 


Figure  2.3-1  Sample  Life  Cycle  Activity  Text  Frame 


2.4  GUIDEBOOK  REFERENCE 

Our  third  example  of  use  of  the  Reference  Manual  is  “User  C”  who  would  like  to 
look  up  a  function,  learn  what  attributes  are  associated  with  it,  and  find  evaluation  techniques 
relevant  to  particular  function-attribute  pairs.  User  C  first  consults  the  Function  Index,  Chapter  7. 
A  sample  Function  Index  Frame  [7. 1.6. 7  Compilation]  is  shown  in  Fig.  2.4-1.  Functions  are  re¬ 
lated  to  i it <3  cycle  activities  and  tools.  In  the  example,  the  function,  Compilation,  is  related  to  five 
life  cycle  activity  groups:  Coding  and  CSU  Testing,  CSC  Integration  and  Testing,  CSCI  Testing, 
System  Integration  and  Testing,  and  Operational  Testing  and  Evaluation.  The  user  of  the  manual 
can  refer  to  Chapter  4  to  find  more  information  about  these  activity  groups.  The  Compiler  is 
the  tool  that  performs  the  function  Compilation.  Information  about  compilers  can  be  found  in 
Chapter  5.  User  C  is  interested  in  evaluation  of  the  compilation  function  with  respect  to  various 
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Descriptions  of 
Transformation  Functions 
Useful  in 

“Coding  and  CSU  Testing" 
Phase 


USER  B 


G-06647 


Figure  2.3-2  Example  of  Cross  Reference:  User  B 


attributes.  For  example,  the  function-attribute  pair  Compilation-Processing  Effectiveness  is  rep¬ 
resented  by  the  items  under  Guidebook  References  in  this  text  frame.  The  first  line  of  the  Guide¬ 
book  References  gives  the  attribute  that  is  paired  with  Compilation  to  point  to  a  section  in  the 
Guidebook  describing  a  specific  assessment  technique.  Here,  the  attribute  Processing  Effec¬ 
tiveness  (described  in  Section  6.4.22  of  the  Reference  Manual)  is  paired  with  Compilation.  This 
pair  points  to  sections  in  the  Guidebook  called  IDA  Benchmarks  and  ACEC,  among  others, 
which  provide  additional  information  on  these  two  E&V  techniques.  A  user  of  the  Reference 
Manual  can  find  references  to  these  two  techniques  in  the  Attribute  Index  or  the  Function  Index. 
The  User  C  scenario  is  pictorially  represented  in  Fig.  2.4-2. 
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7.1. 6.7  Compilation 
Description: 

Translating  a  computer  program  expressed  in  a  procedural  or  problem-oriented  language 
into  object  code.  [@Kean  1985] 

Cross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing 

4.6.2, 

CSC  Integration  and  Testing 

4.7.2, 

CSCI  Testing 

4.8.2, 

System  Integration  and  Testing 

4.9.2, 

Operational  Testing  and  Evaluation 

4.10.2] 

Tools: 

[Translating  Editor 

5.12.1, 

Compiler  (Front  End) 

5.12.2, 

Compiler  (Code  Generator  —  Back  End) 

5.12.3] 

Guidebook  References: 


’  [Anomaly  Management 

@GB:  ARTEWG  Catalogue  of  Ada  Runtime 

6.4.2, 

Implementation  Dependencies 

5.10 

Anomaly  Management 

6.4.2, 

@GB:  Compiler  Assessment  Questionnaire 

5.12 

Augmentability 

6.4.4, 

@GB:  Compiler  Assessment  Questionnaire 

5.12 

Capacity 

6.4.6, 

@GB:  IDA  Benchmarks 

5.2; 

Capacity 

6.4.6, 

@GB:  MITRE  Benchmark  Generator  Tool 

5.6; 

Power 

6.4.21, 

@GB:  Compilation  Checklist 

5.8; 

Power 

6.4.21, 

@GB:  Compiler  Assessment  Questionnaire 

5.12; 

'Processing  Effectiveness 

6.4.22, 

@GB:  IDA  Benchmarks 

5.2; 

'Processing  Effectiveness 

6.4.22, 

@GB:  ACEC 

5.3: 

'Processing  Effectiveness 

6.4.22, 

@G8:  PIWG  Benchmark  Tests 

5.4; 

'Processing  Effectiveness 

6.4.22, 

@GB:  University  of  Michigan  Benchmark  Tests 

5.5: 

'Processing  Effectiveness 

6.4.22, 

@GB:  ARTEWG  Catalogue  of  Ada  Runtime 
Implementation  Dependencies 

5.10; 

Processing  Effectiveness 

6.4.22, 

@GB:  Compiler  Assessment  Questionnaire 

5.12; 

Proprietary  Rights 

6.4.23, 

@GB:  Compiler  Assessment  Questionnaire 

5.12; 

Figure  2.4-1  Sample  Functions  Index  Frame 
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2.5  REFERENCE  FRAMEWORK 

Figure  2.5-1  depicts  the  Classification  Schema  as  an  internal  framework  for  the  Ref¬ 
erence  Manual.  The  schema  provides  paths,  within  and  between  indexes,  that  users  can  follow 
to  extract  information  directly  or  to  find  sections  in  the  Guidebook  that  describe  elements  of  E&V 
technology.  The  figure  also  indicates  the  direct  relationships,  that  is,  types  of  cross  references 
and  Guidebook  references  featured  in  the  Reference  Manual. 

The  Function  Index  is  seen  to  be  directly  related  to  all  of  the  other  indexes.  Thus,  it 
is  drawn  in  an  “anchoring"  position  in  the  diagram.  Attributes,  also,  play  a  key  role  in  the  overall 
E&V  process,  in  two  ways.  First,  many  assessment  objectives  are  defined  in  terms  of  function- 
attribute  pairs,  such  as  compilation-efficiency  and  editing-power.  Second,  other  attributes 
(those  not  “pairing-up"  with  functions)  represent  factors  or  criteria  by  which  APSEs  or  APSE 
components  are  assessed,  independent  of  the  functions  performed.  Examples  of  the  latter  are: 
maintainability,  interoperability,  maturity,  and  cost. 
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USERS 


AND  TECHNIQUES 
DESCRIBED  IN  THE 
GUIDEBOOK 


Figure  2.5-1  The  Schema  as  Framework  for  the  Reference  Manual 


Besides  the  direct  relationships  indicated  in  Fig.  2.5-1,  indirect  relationships  can 
also  be  useful  and  are  constructed  by  combining  two  or  more  direct  relationships.  For  example, 
since  activities  and  functions  are  related  and  functions  and  tools  are  related,  it  is  possible  to  de¬ 
termine  the  relationships  between  activities  and  tools.  It  should  be  noted  that  not  all  such  con¬ 
structions  will  be  useful.  Such  a  useless  relationship  might  be  life  cycle  activities  and  attributes 
related  via  function. 


The  conceptual  structure  pictured  in  Fig.  2.5-1  has  an  open-ended  quality  in  several 
ways.  First,  there  is  no  fixed  number  of  indexes;  more  can  be  added  as  desired.  As  each  new 
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index  is  added  a  new  section  of  the  Reference  Manual  can  be  added,  providing  appropriate  defi¬ 
nitions  and  cross-references  to  other  indexes.  The  new  index  would  be  represented  by  an  addi¬ 
tional  block  along  the  “main  diagonal.”  Second,  each  individual  index  may  have  elements 
added  to  it  as  new  understanding  of  the  various  aspects  of  APSEs  and  the  E&V  process  is 
gained.  The  process  of  modifying  the  structure  of  an  individual  index  in  this  way  would  take 
place  within  the  boxes.  Finally,  the  off-diagonal  space,  above  and  below  the  main  diagonal,  rep¬ 
resents  the  notion  that  any  two-way  relationship  may  be  included  in  the  system.  Thus,  any  sec¬ 
tion  may  be  referenced  by  other  sections,  and  any  section  may  reference  other  sections.  Also, 
any  two  indexes  may  be  involved  in  a  relationship  of  the  type  that  defines  an  E&V  objective  or 
points  to  an  E&V  technique  in  the  Guidebook.  Although  many  of  the  potential  combinations  are 
not  expected  to  be  useful  or  relevant  to  E&V  purposes,  the  structure  permits  the  consideration  of 
all  possibilities. 

The  procedures  described  in  the  Guidebook  are  organized  into  chapters,  as  indi¬ 
cated  in  Fig.  2.5-2.  Chapters  4  and  beyond  are  “formal  chapters”  built  on  a  standard  format 
(see  Guidebook) .  Most  of  the  pointers  from  the  Reference  Manual  to  the  Guidebook  refer  to  spe¬ 
cific  instances  of  evaluation  (E)  or  validation  (V)  techniques  described  in  these  formal  chapters. 
There  are  also  some  “backward  references”  from  these  chapters  to  Chapter  4,  “Synopses” 
[@GB  4]  or  to  the  Reference  Manual. 

Evaluation  techniques  can  be  characterized  as  based  on  subjective  judgments  or 
objective  procedures,  designed  to  assess  performance  and  quality.  Validation  techniques  can 
be  characterized  as  based  on  informal  or  formal  procedures,  designed  to  assess  conformance 
to  a  standard.  The  types  of  E&V  techniques  described  in  the  Guidebook  fall  into  the  following 
categories: 


•  Questionnaire 

•  Capabilities  Checklist 

•  Assessment  Checklist 

•  Test  Suite 

•  Automated  Test  Suite 

•  Structured  Experiment. 

Examples  of  all  of  the  above  are  spread  throughout  the  Guidebook.  Chapter  3  of  the  Guidebook 
[@GB  3]  provides  a  discussion  of  how  to  combine  results  from  a  variety  of  techniques  to  perform 
an  integrated  whole-APSE  assessment. 
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E&V  GUIDEBOOK 

1.  Introduction 

2.  Structure  and  Use  of  the  Guidebook 

3.  Integration  of  E&V  Technology 

4.  Synopses _ 

5.  Compilation  System  Assessors 

6.  Target  Code  Generation  Aids  and  Analysis 
Assessors 

7.  Test  System  Assessors 

8.  Tool/Host  Interface  Assessors 

9.  Requirements/Design  Support  Assessors 

10.  Configuration  Management  Support  Assessors 

11.  Distributed  System  Development  and  Runtime 
Support  Assessors 

12.  Distributed  APSE  Assessors 

13.  “Whole  APSE”  Assessors 

14.  Adaptation  Assessors 

15.  Information  Management  Support  Assessors 
99.  Other  Assessors 


E&V  TECHNIQUES 


Figure  2.5-2  Organization  of  the  Guidebook 
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3.  WHOLE  APSE  ASSESSMENT  ISSUES 


This  chapter  is  intended  as  a  starting  place  for  those  interested  in  selecting  or  as¬ 
sessing  an  APSE  considered  as  a  “whole  entity”  that  is  "more  than  the  sum  of  its  parts.”  The 
chapter,  therefore,  cannot  be  organized  in  the  index/text  frame  style  of  other  chapters  in  this 
manual,  because  that  style  fits  the  view  of  an  APSE  as  a  collection  of  components  or  functions 
that  can  be  evaluated  one  at  a  time  and  “added  up.”  Here  we  take  the  opposite  view  —  that  the 
APSE  is  a  total  system  which  serves  a  project  team  across  an  entire  software  development  life 
cycle  —  and  that  it  should  be  evaluated  in  terms  of  overall  project  goals  and  team  productivity, 
rather  than  in  terms  of  individual  atomic  functions. 

Although  the  technology  of  “whole  APSE  assessment”  is  (in  the  late  1980s)  very 
immature,  there  are  many  helpful  materials  in  the  open  literature.  This  chapter  may  be  consid¬ 
ered  a  guide  to  that  literature.  It  is  organized  to  address  the  following  types  of  questions: 

What  is  an  APSE? 

How  can  whole  APSEs  be  viewed? 

What  are  the  key  whole  APSE  attributes? 

How  can  whole  APSEs  be  assessed? 

Where  can  relevant  information  be  found? 

The  first  four  questions  are  addressed  in  Sections  3.1  through  3.4,  respectively.  Various  refer¬ 
ences  to  other  sections  of  the  E&V  Reference  Manual  (RM),  the  E&V  Guidebook  (GB),  and 
open-literature  sources  are  distributed  throughout  the  chapter. 


3.1  APSE  DEFINITIONS  AND  ALTERNATIVE  NAMES 

The  acronym  APSE  stands  for  “Ada  Programming  Support  Environment."  The  term 
“Programming  Support  Environment”  is  defined  in  the  “IEEE  Standard  Glossary  of  Software  En¬ 
gineering  Terminology”  [@IEEE  1983]  as: 
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“An  integrated  collection  of  tools  accessed  via  a  single  command  language  to 
provide  programming  support  capabilities  throughout  the  software  life-cycle. 

The  environment  typically  includes  tools  for  design,  editing,  compiling,  loading, 
testing,  configuration  management,  and  project  management.” 

Thus,  one  useful  definition  of  an  APSE  is  the  above  quotation  along  with  the  stipulation  that  there 
be  at  least  one  Ada  compiler  among  the  tools  provided.  A  similar  definition  for  the  term  IPSE, 
which  stands  for  “Integrated  Project  Support  Environment,”  is  given  [@Lehman  1987]  as: 

“An  embodiment  of  software  technology  in  a  collection  of  tools  for  capture, 
representation,  control,  refinement,  transformation,  and  other  manipulation  of 
project  related  information." 

The  second  name  and  definition  are  broader  than  the  first  because  they  refer  to  total  “project” 
support  and  information,  rather  than  “programming”  support  and  programs.  The  distinction  be¬ 
tween  the  two  is  highlighted  not  only  in  the  Lehman  and  Turski  paper,  but  in  many  others,  includ¬ 
ing  a  survey  paper  by  Houghton  and  Wallace  [@Houghton  1987].  The  latter  uses  the  following 
terms  to  characterize  most  existing  environments: 

•  Framing  Environments 

•  Programming  Environments 

•  General  Environments. 

Framing  environments  concentrate  on  the  early  stages  of  the  life  cycle  and  tend  to  be  methodol¬ 
ogy-specific.  Programming  environments  concentrate  on  the  latter  part  of  the  life  cycle  and  are 
oriented  toward  programming,  debugging,  and  testing.  General  environments  contain  basic 
tools  that  support  all  phases  of  the  life  cycle  and  tend  to  be  methodology-free. 

Both  of  the  above  definitions  and  discussion  are  limited  in  that  they  are  tied  to  one, 
traditional  view  of  an  APSE.  In  this  view  an  APSE  (or  IPSE)  is  seen  as  a  collection  of  tools.  The 
following  section  presents  additional  views,  any  of  which  may  become  increasingly  important  to 
those  who  wish  to  select  or  evaluate  APSEs  of  the  future. 

Another  taxonomic  classification  of  environments  is  presented  in  a  paper  by  Dart, 
Ellison,  Feiler,  and  Habermann  [@Dart  1987].  The  categories  are: 
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•  Language-centered  environments 

•  Structure-oriented  environments 

•  Toolkit  environments 

•  Method-based  environments. 

The  significance  of  the  taxonomy,  according  to  the  authors,  “is  in  its  clarification  of  trends  rather 
than  in  categorization  of  particular  environments.  A  particular  environment  may  fit  a  number  of 
categories,”  (see  discussion  of  views  of  an  APSE  in  the  following  section). 

Additional  acronyms  found  in  the  literature  on  this  same  general  topic  are  listed  be¬ 
low.  among  others  mentioned  previously. 

APSE  —  Ada  Programming  Support  Environment 
IPSE  —  Integrated  Project  Support  Environment 
PSE  —  Programming  Support  Environment 
SDE  —  Software  Development  Environment 
SEE  —  Software  Engineering  Environment 

The  term  SDE.  for  example,  has  been  used  in  the  title  of  several  ACM-sponsored  conferences, 
and  is  the  subject  of  an  IEEE  tutorial  [@!EEE  1981].  All  of  the  above  are  possible  kev  words,  un¬ 
der  which  may  be  found  useful  material  on  whole  APSEs  or  whole  APSE  E&V,  the  subject  of  this 
chapter. 


A  definition  of  an  APSE,  based  on  the  concept  of  a  collection  v.  tools,  is  limited  in 
the  sense  that  it  describes  only  a  portion  of  the  total  environment  for  software  development.  As 
discussed  in  the  paper  "The  Ecology  of  Software  Environments”  [@Wasserman  1981a],  the  to¬ 
tal  environment  or  “surroundings"  includes  such  things  as  the  computer  itself,  peripheral  stor¬ 
age  and  display  devices,  project  management  practices,  organizational  characteristics,  and 
external  constraints  such  as  governmental  directives  and  physical  workspace  factors.  All  of  the 
above  will  influence  the  development  and  application  of  whole  APSE  evaluation  techniques. 


3.2  VIEWS  OF  AN  APSE 

Various  ways  of  viewing  an  APSE  are  summarized  briefly  below.  The  views  differ  in 
terms  of  both  how  an  APSE  is  seen  —  that  is,  what  sort  of  conceptual  model  does  the  viewer  hold 
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—  and  by  whom  is  the  APSE  viewed  —  for  example,  the  APSE  user  would  typically  have  a  differ¬ 
ent  view  than  the  APSE  builder.  The  views  presented  are  not  necessarily  mutually  exclusive;  an 
individual’s  view  of  a  specific  APSE  may  combine  aspects  of  several  of  the  following  notions. 

3.2.1  APSE  Viewed  as  a  Collection  of  Tools 

This  is  the  traditional  view  of  programming  support  environments,  and  is  consistent 
with  the  standard  definition  quoted  in  the  previous  section.  An  advantage  of  taking  this  view  is 
that  it  permits  the  viewer  to  characterize  an  APSE  in  terms  of  elements  of  a  functional  taxonomy, 
such  as  that  provided  in  Chapter  7  [Functions  7].  The  characterization  can  be  stated  in  terms  of 
yes/no  answers  to  a  long  list  of  clearly  defined  questions  —  is  function  x.y.z  provided  or  is  it  not? 
A  disadvantage  of  this  view  is  that  it  may  cause  the  viewer  to  neglect  the  crucial  whole-APSE  is¬ 
sues  that  are  the  subject  of  this  chapter,  and  which  are  impossible  to  express  in  terms  of  a  com¬ 
position  of  individual  low-level  functions. 

One  version  of  this  view  is  given  in  the  Stoneman  Report  [@DoD  1980],  which  pic¬ 
tures  a  multi-layered,  extensible  collection  built  around  an  inner  kernel  (KAPSE)  and  a  surround¬ 
ing  minimal  (MAPSE)  layer  of  essential  tools,  as  shown  in  Fig.  3.2-1.  This  version  may  have 
more  relevance  for  APSE  builders  than  for  APSE  users,  since  users  need  not  necessarily  be 
aware  of  the  layer  in  which  a  particular  tool  resides. 

3.2.2  APSE  Viewed  as  a  Methodology-Support  System 

In  this  view  an  APSE  is  seen  as  a  system  that  supports  a  particular  development 
methodology  or  a  particular  model  of  the  software  development  process.  It  might  be  strongly 
tied  to  a  standard  set  of  products  and  activities  such  as  those  required  by  the  U.S.  Department  of 
Defense  [@DoD-STD-2167A]  for  mission  critical  software.  It  might  be  based  upon  a  coherent 
software  development  and  maintenance  methodology,  such  as  that  described  in  “Life-Cycle 
Support  in  the  Ada  Environment"  [@McDermid  1984].  The  latter  expresses  the  view  that,  “The 
purpose  of  a  tool  is  to  support  a  method  by  automating  some  aspect  of  the  use  or  application  of 
the  method.” 

3.2.3  APSE  Viewed  as  an  Information  Management  System 

In  this  view  primary  emphasis  is  placed  on  how  project  information  is  stored, 
retrieved,  manipulated,  and  maintained  over  the  entire  life  cycle.  A  key  issue  is  the  set  of 
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Figure  3.2-1  Stoneman  APSE  Architecture 

interfaces  between  tools  and  the  project  database  [©Houghton  1987].  The  database  may  be 
considered  a  tool  that  is  used  by  other  tools;  in  this  case,  the  database  itself  is  the  interface  be¬ 
tween  the  other  tools.  Another  key  issue  is  the  control  of  access  to  information,  and  how  this  is 
affected  by  data  structure  models  —  such  as  hierarchical,  relational,  transactional,  etc.  McDer- 
mid  [@McDermid  1985]  envisions  three  generations  of  environments  as  follows.  The  first  gen¬ 
eration  would  be  a  conventional  collection  of  Ada  tools  tied  to  a  Unix-like  environment.  The 
second  generation  would  be  based  on  a  database  schema  that  provides  appropriate  life  cycle 
support,  but  has  a  static,  non-adaptive  structure.  The  third  generation  would  provide  a  dynamic, 
adaptive  information  management  system  that  can  encapsulate  any  underlying  database 
schema. 

Another  key  set  of  issues  to  be  addressed  are  those  specific  to  the  Ada  compilation 
process.  A  unique  feature  of  the  Ada  language  is  that  inter-unit  dependencies  exist  at  compila¬ 
tion  time.  Source  code  units  can  be  compiled  separately,  but  not  independently.  The  implica¬ 
tions  of  this  feature  are  discussed  further  in  Section  3.3.1  under  Integrity. 
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3.2.4  APSE  Viewed  as  a  User-Oriented,  Interactive  System 

This  view  emphasizes  user  interfaces  and  human  factors.  According  to  the  authors 
of  a  survey  paper  [©Houghton  1987],  “Many  systems  that  measure  poorly  in  terms  of  human 
factors,  including  most  traditional  operating  systems,  have  withstood  the  test  of  time. .  .  .  The 
real  users  are  the  systems  programmers  or  gurus.'*  (But,  in  the  future)  "Software  engineering 
environments  . . .  should  not  require  a  system  expert  as  an  interface  between  the  software  engi¬ 
neer  and  the  environment."  A  conference  overview  paper  [@Henderson  1987]  sees  “a  trend  to¬ 
ward  the  use  of  graphics.”  The  desire  for  graphical  interactive  workstations  is  echoed  in  a  paper 
on  personal  development  systems  [@Gutz  1981]  with  the  words,  “time  sharing  is  an  idea  whose 
time  is  gone.”  An  important  set  of  issues  concerns  the  different  roles  of  various  users  of  the  sys¬ 
tem,  different  modes  of  use  for  each,  and  how  the  system  orients  its  support  to  the  current  user/ 
mode.  An  influential  set  of  concepts  has  come  from  the  world  of  artificial  intelligence,  such  as 
the  “incremental  enrichment”  style  of  LISP  program  development  [@Barstow  1981]. 

3.2.5  APSE  Viewed  as  a  Knowledge-Based  Expert  System 

In  this  view  components  of  an  APSE  (or  conceivably  the  entire  APSE)  are  created 
as  expert  systems  based  on  the  past  experience  of  human  experts  in  specific  domains.  One  is¬ 
sue  of  IEEE  Expert  [@IEEE  Expert  1986]  contains  a  collection  of  papers  which  address  this  pos¬ 
sibility.  A  paper  in  this  collection  [©Zualkernan  1986]  outlines  some  steps  to  determine  the 
feasibility  of  this  approach  and  treats  the  specific  area  of  software  testing  as  a  case  study.  This 
view  may  be  considered  a  special,  advanced  case  of  the  preceding  view,  where  the  style  of 
user-interaction  is  that  of  an  "expert  assistant.” 

3.2.6  APSE  Viewed  as  a  Stable  Framework 

This  view  has  been  advocated  by  C.M.  McKay  of  U.  Houston  Clear  Lake  [@McKay 
1987],  and  is  motivated  by  the  need  to  support  large,  complex,  non-stop,  distributed,  long-lived 
systems  such  as  the  NASA  Space  Station.  The  APSE  is  viewed  as  a  stable  framework  to  which 
new  or  improved  tools  can  be  added  over  time  without  interfering  with  or  invalidating  previous 
work.  The  framework  is  defined  in  terms  of  standard  phases  and  deliverables,  in  one  dimension, 
and  stable  interface  sets  separating  tools  and  various  classes  of  objects,  in  the  other  dimension. 
Ideally,  the  stability  of  the  framework  and  its  interface  sets  will  allow  both  the  flight  system  and  its 
support  environment  to  evolve  incrementally  as  reliable,  maintainable  systems. 
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Another  view,  similarly  motivated,  is  illustrated  in  Fig.  3.2-2.  This  figure  has  been 
adapted  from  the  “STARS  Consolidated  Development  Plan”  [@STARS  1988];  it  was  supplied  to 
the  E&V  Task  by  Unisys,  one  of  the  three  STARS  prime  contractors.  It  depicts  an  adaptable  en¬ 
vironment  architecture  based  on  layered  portable  interfaces.  The  top  interface  of  the  Virtual 
Layer,  for  example,  is  intended  to  be  independent  of  the  underlying  hardware.  It  could  be  speci¬ 
fied  as  the  CAIS-A  standard,  the  IEEE  POSIX/Ada  interface,  or  an  interface  specifically  engine¬ 
ered  for  the  STARS  program.  Components  of  the  Object  Structure  Layer  use  the  services  of  the 
Virtual  Layer.  Many  of  these  components  will  make  use  of  industry  standard  interfaces  such  as 
GKS,  X,  and  SQL.  Standardization  of  these  interface  layers  permit  the  APSE  to  evolve  in  a  stable 
way. 
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3.3  KEY  ATTRIBUTES  OF  WHOLE  APSES 

This  section  provides  a  brief  discussion  of  what  appear  to  be  some  of  the  key 
whole-APSE  attributes.  The  discussion  is  organized  in  accordance  with  the  attribute  taxonomy 
[Attributes  6]  given  in  Chapter  6  of  this  manual,  which  employs  three  top-level  “acquisition  con¬ 
cern"  categories: 

•  Performance  Attributes  (6.1) 

•  Design  Attributes  (6.2) 

•  Adaptation  Attributes  (6.3). 

Under  each  of  these  is  a  second-level  set  of  “quality  factors",  such  as  Efficiency,  Integrity,  etc. 
These  are  further  decomposed  into  “criteria”,  some  of  which  apply  to  more  than  one  quality  fac¬ 
tor.  All  the  criteria  attributes  are  listed  alphabetically  in  Section  6.4  of  the  Attributes  Index. 

The  definitions  given  in  Chapter  6  are  very  component-oriented.  For  example,  the 
definition  of  Efficiency  [Efficiency  6.1.1]  includes  the  words,  “The  extent  to  which  a  component 
fulfills  its  purpose  using  a  minimum  of  computing  resources.”  A  whole-APSE  version  of  the  defi¬ 
nition  of  Efficiency  might  read,  “The  extent  to  which  an  APSE  supports  life  cycle  activities  using  a 
minimum  of  computing  and  development-team  resources.”  This  same  kind  of  language  modifi¬ 
cation  is  used  throughout  the  discussion  to  follow. 

The  selection  of  attributes  highlighted  below  has  been  influenced  by  discussions 
within  the  E&V  Team  and  by  the  following  papers  (some  of  which  make  reference  to  many  other 
papers):  “Toward  Integrated  Software  Development  Environments”  [@Wasserman  1981b], 
“Software  Development  Environment  Issues  as  Related  to  Ada”  [@Notkin  1981],  “Essential 
Properties  of  IPSEs"  [@Lehman  1987],  and  “Characteristics  and  Functions  of  Software  Engi¬ 
neering  Environments:  An  Overview”  [@Houghton  1987]. 

A  quotation  from  another  paper  by  Wasserman  [@Wasserman  1981a]  perhaps 
best  sets  the  stage  for  this  discussion  of  key  whole-APSE  attributes.  He  says,  “The  goal  is  to 
create  an  environment  that  not  only  enhances  developer  productivity  but  also  supports  the  crea¬ 
tion  of  superior  products.” 

3.3.1  Performance  Attributes 

rEfficiencv  6.1.1]  —  As  stated  above,  efficiency,  in  the  whole-APSE  context,  is 
measured  in  terms  of  resources  used  by  an  entire  team  in  performing  an  entire  group  of  activities 
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or  major  “chunk"  of  work  during  development  of  a  software  product.  This  attribute  is  clearly  re¬ 
lated  to  the  overall  project  goal  of  team  productivity. 

flntearitv  6.1.2]  —  This  attribute  deals  with  the  extent  to  which  access  to  the  soft¬ 
ware  environment  or  other  data  is  controlled,  especially  for  the  purposes  of  monitoring  status, 
preserving  integrity  of  different  versions,  and  controlling  changes.  It  is  an  attribute  of  an  impor¬ 
tant  set  of  management  functions  that  greatly  influence  team  productivity  and  product  quality.  A 
critical  aspect  of  this  attribute  concerns  the  treatment  of  the  unique  Ada  feature  (discussed  in 
Section  3.2.3)  associated  with  inter-unit  dependencies.  In  managing  Ada  program  library  units 
strict  rules  apply  to  the  order  of  compilation.  As  a  project  is  developed,  obsolete  units  need  to  be 
recompiled,  compilation  order  changes,  and  Ada  closure  sets  may  be  redefined.  Therefore,  the 
APSE’s  management  of  Ada  source  and  object  modules  should  be  assessed  with  these  special 
requirements  in  mind. 

rUsability/Anomalv  Management  6. 1.5/6. 4.2] 
and 

fUsabilitv/Cost  6.1.5/6.4.11] 
and 

fUsability/Maturitv  6.1.5/6.4.18]  —  These  are  the  aspects  of  usability  that  naturally 
concern  managers  and  controllers  of  facility  investment  resources.  Anomaly  management  af¬ 
fects  the  probability  that  an  APSE  will  be  functionally  ready  at  some  specified  point  in  time.  It 
must  operate,  of  course,  on  available  hardware.  Absence  of  such  availability  in  a  timely  way 
would  naturally  be  disastrous. 

Cost  includes  the  cost  of  purchase  or  lease,  installation,  user  assistance,  and  main¬ 
tenance.  Its  importance  is  self-evident. 

Maturity  is  the  extent  to  which  an  APSE  has  been  used  in  the  development  of  deliv¬ 
erable  software  by  typical  users  and  to  which  the  feedback  from  that  use  has  been  reflected  in 
improvements. 

rUsabilitv/Ooerabilitv  6.1.5/6.4.20] 
and 

fUsabilitv/T raining  6.1.5/6.4.36]  —  These  represent  the  human-factors  aspects 
of  usability  such  as  ease-of-use,  ease-of-learning,  on-line  help  features,  and  consistency  of 
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interfaces.  This  set  of  attributes  will  have  a  major  influence  not  only  on  long-term  productivity, 
but  on  the  early  acceptance  of  an  APSE  by  individuals  and  the  team  as  a  whole.  The  resulting 
impact  on  motivation  and  team  spirit  can  be  crucial. 

3.3.2  Design  Attributes 

ICorrectness/ComDleteness  6. 2.1/6. 4.9]  —  This  is  the  extent  to  which  an  APSE 
supports  the  complete  set  of  operations  necessary  to  perform  its  intended  function,  which  is  to 
provide  full  support  to  a  development  team  across  an  entire  product  life  cycle.  This  attribute 
could  be  interpreted  in  a  minimal  way,  listing  only  truly  essential  (MAPSE)  functions  necessary 
for  a  particular  project.  Alternatively,  it  could  be  interpreted  as  a  list  of  functions  desired  to  pro¬ 
vide  the  capability  to  produce  high-quality  products. 

fMaintainabilitv/Self-Descriotiveness  6.2.2/6.4.28]  —  In  the  whole-APSE  context  the 
aspect  of  this  attribute  to  be  emphasized  concerns  the  underlying  database  or  schema  used  to 
store  and  retrieve  project  data.  In  one  of  the  papers  cited  above  [@Lehman  1987]  this  attribute 
was  called  data-structuredness.  The  Ada-Europe  document  “Selecting  an  Ada  Environment” 
[@Lyons  1986]  says,  “It  is  now  generally  recognized  that  the  totality  of  information  that  a  project 
must  store,  consists  not  only  of  individual  entities  containing  data  but  also  of  the  relationships  be¬ 
tween  them,  such  as  the  facts  that  a  particular  object  file  has  been  compiled  from  a  particular 
source  file,  that  a  source  file  implements  a  particular  specification,  or  that  an  error  report  relates 
to  a  particular  release  of  a  system." 

rVerifiability,  Testability  6.2.3]  —  This  is  the  extent  to  which  an  APSE  facilitates 
evaluation  of  its  own  performance,  so  that  productivity  and  quality  metrics  can  be  gathered  and 
analyzed.  It  is  through  the  use  of  this  attribute  that  the  extensibility  (see  below)  of  an  APSE  can 
be  exploited  effectively  in  a  continual  process  of  improvement. 

3.3.3  Adaptation  Attributes 

fExpandabilitv/Auamentabilitv  6.3.1/6.4.4]  -  This  is  the  extent  to  which  an  APSE 
facilitates  the  addition  of  new  capabilities  in  response  to  needs  that  go  beyond  its  original  re¬ 
quirements.  The  new  capabilities  could  include  new  functions,  expanded  data  capacity,  or  new 
types  of  data  relationships.  The  word  used  to  describe  this  feature  in  several  of  the  papers  cited 
above  is  extensibility.  Given  the  current  state  of  the  art  and  rapid  pace  of  change  of  environment 
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technology,  the  presence  of  this  quality  (however  it  is  achieved)  appears  necessary  in  order  to 
provide  a  path  for  evolutionary  improvements. 

flnteroperabilitv/Commonalitv  6. 3. 2/6. 4.7]  —  This  is  the  ability  of  APSEs  to  ex¬ 
change  database  objects  and  their  relationships  without  conversion  of  formats,  and  the  use  of  in¬ 
terface  standards  (e.g.,  CAIS  [@CAIS])  to  facilitate  such  exchanges. 

fTransportabilitv  6.3.4]  —  This  is  the  extent  to  which  an  APSE  supports  the  move¬ 
ment  of  software  components  to  or  from  another  APSE  without  change  in  functionality  or 
reprogramming,  and  the  use  of  interface  standards  (e.g.,  CAIS)  to  facilitate  such  movements. 
Another  whole-APSE  attribute  that  may  be  important  is  the  extent  to  which  the  entire  APSE  can 
be  transferred  to  a  different  host/operating  system,  based  on  its  use  of  standardized  interfaces  to 
the  underlying  system. 


3.4  APPROACHES  TO  WHOLE-APSE  E&V 

This  section  provides  a  brief  discussion  of  approaches  to  whole-APSE  assessment, 
including  both  evaluation  of  performance  and  quality,  and  validation  of  conformance  to  stan¬ 
dards.  References  to  Guidebook  sections  and  other  documents  are  included,  in  cases  where 
there  are  known  examples  of  existing  assessors  or  assessment  products  under  development. 
Such  assessors  can  be  categorized  generally  as  either  “tools”  or  “aids."  [@E&V  Tools  and 
Aids  1987]  A  more  refined  breakdown  is  the  following: 

•  Benchmarks  and  Test  Suites  (Tools) 

•  Checklists  and  Questionnaires  (Aids) 

•  Structured  Experiments  (Aids  that  may  use  Tools) 

•  Decision  Aids  (Aids  that  may  use  Tools  and  other  Aids). 

These  are  discussed,  in  turn,  in  the  following  four  subsections. 

3.4.1  Benchmarks  and  Test  Suites 

Benchmarks  are  standard  tests  used  to  measure  the  execution,  performance  or  ac¬ 
ceptability  of  an  APSE  function  or  set  of  functions.  A  test  suite  is  an  organized  collection  of  such 
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tests.  The  Ada  Compiler  Validation  Capability  [@ACVC  1989]  is  a  test  suite  designed  to  test 
conformance  of  an  Ada  compiler  to  the  formal  definition  of  the  language  [@DoD  1983] .  A  proto¬ 
type  compiler  performance  evaluation  test  suite  was  generated  by  the  Institute  for  Defense 
Analysis  [@GB:  5.2],  and  a  more  carefully  engineered  set  known  as  the  ACEC  or  Ada  Compiler 
Evaluation  Capability  [@GB:  5.3]  has  been  developed,  under  an  E&V  Task  contract,  by  the 
Boeing  Company.  Another  collection  of  Ada  compiler  performance  tests  has  been  gathered  by 
the  ACM  SIGAda  Performance  Issues  Working  Group  [@PIWG  1987] .  Results  of  these  tests,  run 
on  a  number  of  commercial  compilers,  will  be  published  in  the  open  literature. 

3.4.2  Checklists  and  Questionnaires 

Checklists  and  questionnaires  are  used  to  gather  data  from  vendors  or  users  of 
tools  and  APSEs.  Examples  of  such  data  might  include  specification  parameters,  design  fea¬ 
tures,  historical  information,  typical  usage  scenarios,  implementation  strategies,  enhancement 
plans  or  desires,  and  problem  reports.  An  early  example,  in  Ada  terms,  of  such  a  questionnaire 
was  one  applied  to  the  evaluation  of  the  ROLM  Ada  Work  Center  [@Castor  1983]. 

A  whole-APSE-oriented  example  of  the  use  of  questionnaires  is  the  book  “Selecting 
an  Ada  Environment”  [©Lyons  1986],  which  is  synopsized  in  the  E&V  Guidebook  [@GB:  4.9]. 
The  book  is  organized  around  background  discussions  of  various  topics  followed  by  appropriate 
questions  addressing  each  topic.  The  E&V  Guidebook  [@GB  13.1]  provides  an  example  of  a 
single  APSE  Characterization  Form  that  can  be  used  to  capture  and  organize  a  summary  of  the 
capabilities  and  features  of  an  APSE.  The  Guidebook  also  contains  many  examples  of  check¬ 
lists  devoted  to  individual  topics. 

3.4.3  Structured  Experiments 

Structured  experiments,  based  on  model  projects  involving  an  aggregation  of  APSE 
functions  or  tools,  can  be  performed  using  APSEs  or  APSE  components  to  gather  data  in  a  sys¬ 
tematic  and  controlled  manner.  These  experiments  can  be  used  for  both  qualitative  and  quanti¬ 
tative  assessments  of  the  functionality,  usability,  and  performance,  as  well  as  for  the  more 
informal  characteristics  of  APSEs. 

3.4.4  Decision  Aids 

Decision  aids  allow  a  user  to  assess  an  APSE  from  a  particular  point  of  view.  Deci¬ 
sion  aids  may  combine  the  results  of  a  number  of  tests,  questionnaires,  and/or  monitored  experi¬ 
ments,  each  of  which  is  weighted  according  to  its  value  for  the  view  being  considered  [@GB  3], 
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4. 


LIFE  CYCLE  ACTIVITIES 


This  chapter  deals  with  the  life  cycle  activities  served  by  the  support  environment. 
The  foundation  for  this  discussion  is  DoD-STD-2167A  [@DoD-STD-2167A].  The  divisions  of  the 
life  cycle  that  are  chosen  to  represent  the  progression  of  activities  depend  on  the  view  that  a  pc: 
son  has  of  the  APSE.  If  one  thinks  of  the  APSE  as  only  providing  an  environment  for  developing 
software,  then  the  activity  groups  can  be  limited  to  the  following  six: 


•  Software  Requirements  Analysis 

•  Preliminary  Design 

•  Detailed  Design 

•  Coding  and  CSU  Testing 

•  CSC  Integration  and  Testing 

•  CSCI  Testing. 


However,  if  one  sees  the  APSE  as  providing  support  across  the  entire  system  life 
cycle  (Integrated  Project  Support  Environment,  or  IPSE,  rather  than  APSE)  [Whole  APSE  Issues 
3.],  then  the  following  five  activity  groups  should  be  added  as  well: 

•  System  Concepts 

•  System  Requirements  Analysis/Design 

•  System  Integration  and  Testing 

•  Operational  Testing  and  Evaluation 

•  Change  Requirements. 


Of  the  above  five  activity  groups,  the  first  two  precede  the  six  software  development 
phases  and  the  next  two  follow  the  six  groups  in  the  overall  system  life  cycle.  The  last  activity 
group,  change  requirements,  is  one  that  may  be  started  in  parallel  with  any  of  the  other  groups 
and  addresses  the  issues  of  enhancement,  error  correction,  and  modification. 


Also,  a  global  life  cycle  'activity  group’  is  introduced.  This  is  not  a  true  portion  of  the 
life  cycle,  but  represents  a  group  of  activities  that  mav  occur  anywhere  across  the  entire  life  cv- 
Cifi.  An  example  of  a  global  function  is  general  purpose  text  editing. 
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Chapter  4  of  DoD-STD-2l67A  lists  the  general  requirements  for  the  life  cycle  activi¬ 
ties.  These  requirements  fall  into  the  following  activity  classes: 

•  Software  Development  Management 

•  Software  Engineering 

•  Formal  Qualification  Testing 

•  Software  Product  Evaluations 

•  Software  Configuration  Management 

•  Transitioning  to  Software  Support. 

These  activities  are  used  as  a  second-level  of  classification  under  each  top-level  life  cycle,  ac¬ 
tivity  division  of  this  chapter. 

Although  this  chapter  is  structured  around  the  life  cycle  activities  of  DoD-STD- 
2167A,  the  information  is  equally  applicable  to  software  life  cycles  that  do  not  fit  that  model.  The 
following  may  be  considered  to  be  a  relatively  generic  view  of  the  activities  encompassed  in  a 
software  life  cycle: 

•  Requirements  Engineering 

•  Design  Engineering 

•  Implementation 

•  Maintenance. 

This  is  a  generic  segmentation  since,  regardless  of  the  software  engineering  approach  taken,  it 
is  necessary  to:  1)  decide  what  to  build,  2)  decide  how  to  build  it,  3)  build  it,  and  4)  support  it  for 
its  useful  life.  Different  life  cycle  models  combine  and  structure  these  activities  in  different  ways, 
but  each  activity  is  usually  included.  In  a  sense,  they  are  the  invariants  of  software  engineering. 

Mapping  the  DoD-STD-2l67A  life  cycle  activities  to  the  generic  set  introduced 
above  results  in  the  groupings  shown  in  Table  4-1 .  In  the  generic  list  of  life  cycle  activities,  each 
is  assumed  to  incorporate  the  analysis  of  its  products.  This  can  include  requirements  analysis, 
design  reviews,  code  walkthroughs,  and  dynamic  analysis  or  conventional  testing  activities. 
Therefore,  no  separate  testing  activity  is  listed.  If  an  additional  activity  is  desired  to  highlight  test¬ 
ing  or  integration  activities,  the  mapping  that  results  is  straightforward. 
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Table  4-1  Life  Cycle  Activities  Mapping 


GENERIC  LIFE  CYCLE  ACTIVITIES 

DoD-STD-21 67A  LIFE  CYCLE  ACTIVITIES 

Requirements  Engineering 

System  Concepts 

System  Requirements  Analysis/Design 
Software  Requirements  Analysis 

Design  Engineering 

Preliminary  Design 

Detailed  Design 

Implementation 

Coding  and  CSU  Testing 

CSC  Integration  and  Testing 

CSCI  Testing 

System  Integration  and  Testing 

Operational  Testing  and  Evaluation 

Maintenance 

Change  Requirements 

Regardless  of  the  life  cycle  of  interest  (e.g.,  Spiral  Model  [@Boehm  1988]  or  rapid 
prototyping),  it  should  be  possible  to  map  its  activities  to  the  generic  set  and  then  follow  the  map¬ 
ping  to  DoD-STD-2167A  to  locate  relevant  information  in  this  chapter.  The  second-level  of  clas¬ 
sification  can  be  a  further  guide  to  the  functions  to  be  expected  to  support  management, 
engineering,  testing,  quality  assurance,  configuration  management,  and  preparation  for  mainte¬ 
nance. 


Figure  4-1  shows  the  relationships  between  the  life  cycle  activities  and  the  other 
elements  in  the  Reference  Manual.  Each  of  the  life  cycle  activity  groups  are  described  in  the  fol¬ 
lowing  sections  and  under  each  section  are  six  subsections  that  deal  with  each  of  the  activity 
classes.  The  functions  specified  for  each  section  are  those  that  are  typically  used  in  that  portion 
of  the  life  cycle. 
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Figure  4-1  Life  Cycle  Activity  Relationships 
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4.1  SYSTEM  CONCEPTS 


4.1.1  System  Development  Management 

Cross  References: 

Products: 

Functions: 


4.1.2  System  Engineering 

Cross  References: 

Products: 

Functions: 

[Requirements  Simulation  7.3.2. 1, 

Requirements  Prototyping  7. 3. 2. 2. 

Simulation  and  Modeling  7.3.2. 3] 


4.1.3  Formal  Qualification  Testing 

Cross  References: 

Products: 

Functions: 
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4.1.4  System  Product  Evaluations 

Cross  References: 

Products: 

Functions: 


4.1.5  Configuration  Management 

Cross  References: 

Products: 

Functions: 

4.1.6  Transitioning  to  System  Support 

Cross  References: 

Products: 

Functions: 
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4.2  SYSTEM  REQUIREMENTS  ANALYSIS/DESIGN 

4.2.1  System  Development  Management 

Cross  References: 

Products: 

[Software  Development  Plan  (SDP)] 

Functions: 

[Predefined  and  User-Defined  Forms  7. 1.2.3] 


4.2.2  System  Engineering 

Cross  References: 

Products: 

[System/Segment  Specification  (SSS), 

Prime  Item  Development  Specification  (PIDS), 

Critical  Item  Development  Specification  (CIDS), 

System/Segment  Design  Document  (SSDD), 

Software  Requirements  Specification  (SRS), 

Interface  Requirements  Specification  (IRS)] 

Functions: 

[Predefined  and  User-Defined  Forms  7. 1.2. 3, 

Systems  Requirements  7. 1.6.1, 

Requirements  to  Natural  Language  7. 1.6. 3, 

Quality  Measurement  7.3. 1.9, 

Completeness  Checking  7.3.1.12, 

Consistency  Checking  7.3.1.13, 

Maintainability  Analysis  7.3.1.18, 

Requirements  Simulation  7. 3.2.1, 

Requirements  Prototyping  7.3. 2. 2] 
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4.2.3  Formal  Qualification  Testing 

Cross  References: 

Products: 

Functions: 

[Predefined  and  User-Defined  Forms 
Documentation  Management 
Testability  Analysis 
Consistency  Checking 
Cross  Reference 


4.2.4  System  Product  Evaluations 

Cross  References: 

Products: 

Functions: 

[Consistency  Checking 
Auditing 

4.2.5  Configuration  Management 

Cross  References: 

Products: 

Functions: 


4.2.6  Transitioning  to  System  Support 

Cross  References: 

Products: 

[Computer  Resources  Integrated  Support  Document 
Functions: 

[Predefined  and  User-Defined  Forms 


7.1. 2.3, 
7.2.1. 2. 
7.3.1. 7, 
7.3.1.13, 
7.3.1.17] 


7.3.1.13, 

7.3.1.22] 


(CRISD)] 


7.1. 2.3] 
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4.3  SOFTWARE  REQUIREMENTS  ANALYSIS 

4.3.1  System  Development  Management 

Cross  References: 

Products: 

[Software  Development  Plan  (SDP)] 

Functions: 

[Predefined  and  User-Defined  Forms  7. 1.2. 3, 

Resource  Estimation  7.2. 2.5, 

Tracking  7.2. 2.6] 


4.3.2  Software  Engineering 

Cross  References: 

Products: 

[Software  Requirements  Specification  (SRS), 

Interface  Requirements  Specification  (IRS)] 

Functions: 

[Predefined  and  User-Defined  Forms  7. 1.2. 3, 

Software  Requirements  7. 1.6.2, 

Requirements  to  Natural  Language  7. 1.6.3, 

Program  Library  Management  7.2. 1.7, 

Data  Flow  Analysis  7.3. 1.3, 

Functional  Analysis  7.3. 1.4, 

Traceability  Analysis  7.3. 1.6, 

Quality  Measurement  7. 3. 1.9, 

Maintainability  Analysis  7.3.1.18, 

Requirements  Simulation  7.3. 2.1, 

Requirements  Prototyping  7.3.2. 2] 
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4.3.3  Formal  Qualification  Testing 

Cross  References: 

Products: 

Functions: 

[Predefined  and  User-Defined  Forms  7. 1.2.3, 

Documentation  Management  7.2. 1.2, 

Cross  Reference  7.3.1.17] 

4.3.4  Software  Product  Evaluations 

Cross  References: 

Products: 

Functions: 

[Testability  Analysis  7.3. 1.7, 

Completeness  Checking  7.3.1.12, 

Consistency  Checking  7.3.1.13, 

Cross  Reference  7.3.1.17, 

Auditing  7.3.1.22] 

4.3.5  Configuration  Management 

Cross  References: 

Products: 

Functions: 

[Specification  Management  7.2. 1.6] 

4.3.6  Transitioning  to  Software  Support 

Cross-References: 

Products: 

[Computer  Resources  Integrated  Support  Document  (CRISD)] 

Functions: 

[Predefined  and  User-Defined  Forms  7.1. 2.3] 
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4.4  PRELIMINARY  DESIGN 


4.4.1  Software  Development  Management 

Cross  References: 

Products: 

[Software  Development  Plan 
Functions: 

[Predefined  and  User-Defined  Forms 
Resource  Estimation 
Tracking 


4.4.2  Software  Engineering 

Cross  References: 

Products: 

[Software  Design  Document 
Interface  Design  Document 

Functions: 

[Predefined  and  User-Defined  Forms 
Preliminary  Design 
Design  Generation 
Documentation  Management 
Program  Library  Management 
Interface  Analysis 
Testability  Analysis 
Quality  Measurement 
Complexity  Measurement 
Reusability  Analysis 
Maintainability  Analysis 
Invocation  Analysis 
Structured  Walkthrough 
Auditing 
Type  Analysis 
Units  Analysis 
Simulation  and  Modeling 
Design  Prototyping 


(SDP)] 


7.1. 2.3, 
7. 2. 2. 5, 
7.2.2.6J 


(SDD), 
(IDD)  ] 


7.1. 2.3, 

7.1. 6.4, 

7.1. 7.1, 

7.2.1. 2, 

7.2.1. 7, 

7. 3. 1.5, 

7. 3. 1.7, 

7.3.1. 9, 

7.3.1.10, 
7.3.1.14, 

7.3.1.18, 

7.3.1.19, 

7.3.1.21, 

7.3.1.22, 

7.3.1.27, 

7.3.1.28, 

7. 3. 2. 3, 
7. 3. 2. 4] 
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4.4.3  Formal  Qualification  Testing 


Cross  References: 
Products: 

[Software  Test  Plan 


Functions: 

[Predefined  and  User-Defined  Forms 
Documentation  Management 
Test  Condition  Analysis 


4.4.4  Software  Product  Evaluations 

Cross  References: 

Products: 

Functions: 

[Requirements  Reconstruction 
Completeness  Checking 
Consistency  Checking 
Cross  Reference 
Scanning 
Auditing 


4.4.5  Configuration  Management 

Cross  References: 

Products: 

Functions: 

[Specification  Management 


(STP)] 


7. 1.2. 3, 
7.2. 1.2, 
7.3.1. 8] 


7.1 .7.2, 

7.3.1.12, 

7.3.1.13, 
7.3.1.17, 
7.3.1.20, 
7.3.1.22] 


7.2.1. 6] 
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4.4.6  Transitioning  to  Software  Support 

Cross  References: 

Products: 

[Computer  Resources  integrated  Support  Document  (CRISD), 
Computer  System  Operator’s  Manual  (CSOM), 

Software  User’s  Manual  (SUM)] 

Functions: 

[Predefined  and  User-Defined  Forms  7. 1.2. 3] 
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4.5  DETAILED  DESIGN 


4.5.1  Software  Development  Management 

Cross  References: 

Products: 

[Software  Development  Plan 

(SDP)  ] 

Functions: 

[Predefined  and  User-Defined  Forms 

7. 1.2. 3, 

Resource  Estimation 

7. 2. 2. 5, 

Tracking 

7.2.2.6J 
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4.5.2  Software  Engineering 

Cross  References: 

Products: 

[Software  Design  Document 
Interface  Design  Document 
Software  Development  File 

Functions: 

[Data  Editing 

Predefined  and  User-Defined  Forms 

Detailed  Design 

Design  Generation 

Database  Management 

Documentation  Management 

File  Management 

Program  Library  Management 

Test  Data  Management 

Interface  Analysis 

Testability  Analysis 

Test  Condition  Analysis 

Quality  Measurement 

Complexity  Measurement 

Reusability  Analysis 

Cross  Reference 

Maintainability  Analysis 

Invocation  Analysis 

Structured  Walkthrough 

Auditing 

Type  Analysis 

Units  Analysis 

Design  Prototyping 

Formal  Verification 


(SDD), 

(IDD). 

(SDF)] 


7.1. 1.2, 

7.1 .2.3, 

7. 1.6. 5, 

7.1 .7.1 , 

7.2.1. 1, 

7.2.1 .2, 

7.2.1. 3, 

7.2.1. 7, 

7.2.1 .8, 

7.3.1. 5, 

7.3.1. 7, 

7.3.1. 8, 

7.3.1. 9, 

7.3.1.10, 
7.3.1.14, 

7.3.1.17, 

7.3.1.18, 

7.3.1.19, 

7.3.1.21, 

7.3.1.22, 

7.3.1.27, 

7.3.1.28, 

7. 3. 2. 4, 
7.3.3] 
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4.5.3  Formal  Qualification  Testing 

Cross  References: 


Products: 


[Software  Test  Description 

(STD)] 

Functions: 

[Predefined  and  User-Defined  Forms 

7.1. 2.3, 

Documentation  Management 

7.2.1. 2. 

Test  Conditional  Analysis 

7. 3. 1.8, 

Cross  Reference 

7.3.1.17] 

4.5.4  Software  Product  Evaluations 

Cross  References: 

Products: 

Functions: 

[Requirements  Reconstruction 

7.1. 7.2, 

Completeness  Checking 

7.3.1.12, 

Consistency  Checking 

7.3.1.13, 

Cross  Reference 

7.3.1.17, 

Scanning 

7.3.1.20, 

Auditing 

7.3.1.22] 

4.5.5  Configuration  Management 

Cross  References: 

Products: 

Functions: 

[Specification  Management 

7. 2. 1.6] 
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4.5.6  Transitioning  to  Software  Support 

Cross  References: 

Products: 

[Computer  Resources  Integrated  Support  Document 
Computer  System  Operator’s  Manual 
Software  User’s  Manual 


(CRISD), 

(CSOM), 

(SUM), 

(SPM), 

(FSM)] 


7. 1.2.3] 
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1 

4.6  CODING  AND  CSU  TESTING 

I 

1 

4.6.1  Software  Development  Management 

m 

Cross  References: 

1 

Products: 

[Software  Development  Plan 

(SDP)] 

f 

Functions: 

[Predefined  and  User-Defined  Forms 

7. 1.2. 3, 

1 

Resource  Estimation 

7. 2. 2. 5, 

Tracking 

7.2.2.6] 

( 

1 

4.6.2  Software  Engineering 

Cross  References: 

1 

Products: 

1 

[Software  Development  File 

(SDF)] 

1 

Functions: 

1 

[Data  Editing 

7.1. 1.2, 

Predefined  and  User-Defined  Forms 

7.1 .2.3, 

■ 

Assembling 

7.1. 6.6, 

1 

Compilation 

7.1. 6. 7, 

Conversion 

7. 1.6. 8, 

g 

Macro  Expansion 

7. 1.6. 9, 

1 

Structure  Preprocessing 

7.1.6.10, 

Body  Stub  Generation 

7.1.6.11, 

■ 

Preamble  Generation 

7.1.6.12, 

1 

Linking/Loading 

7.1.6.13, 

Interpretation 

7.1.6.14, 

Software  and  System  Test  Communications 

7.1.6.15, 

1 

Program  Generation 

7.1. 7. 3, 

Test  Harness  Generation 

7.1. 7.7, 

Database  Management 

7.2.1. 1, 

I 

Documentation  Management 

7.2.1 .2, 

w 

File  Management 

7.2.1. 3, 

6 

Program  Library  Management 

7.2.1. 7, 

1 
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4.6.2  Software  Engineering  (Continued) 

Functions: 


Test  Data  Management 

7.2.1. 8, 

Data  and  Error  Logging 

7.2.1.11, 

Status  Display 

7.2.1.12, 

I/O  Support 

7.2. 3.2, 

Runtime  Environment 

7.2. 3. 5, 

Comparison 

7.3.1. 1, 

Interface  Analysis 

7.3.1. 5, 

Testability  Analysis 

7.3.1. 7, 

Test  Condition  Analysis 

7.3.1. 8, 

Quality  Measurement 

7.3.1. 9, 

Complexity  Measurement 

7.3.1.10, 

Correctness  Checking 

7.3.1.11, 

Completeness  Checking 

7.3.1.12, 

Consistency  Checking 

7.3.1.13, 

Reusability  Analysis 

7.3.1.14, 

Syntax  and  Semantics  Checking 

7.3.1.15, 

Reachability  Analysis 

7.3.1.16, 

Cross  Reference 

7.3.1.17, 

Maintainability  Analysis 

7.3.1.18, 

Structured  Walkthrough 

7.3.1.21, 

Auditing 

7.3.1.22, 

Error  Checking 

7.3.1.23, 

Statistical  Analysis 

7.3.1.24, 

Statistical  Profiling 

7.3.1.25, 

Structure  Checking 

7.3.1.26, 

Type  Analysis 

7.3.1.27, 

Units  Analysis 

7.3.1.28, 

I/O  Specification  Analysis 

7.3.1.29, 

Sizing  Analysis 

7.3.1.30, 

Data  Reduction  and  Analysis 

7.3.1.31, 

Debugging 

7. 3. 2. 5, 

Executable  Assertion  Checking 

7. 3. 2. 6, 

Constraint  Evaluation  (Contention) 

7. 3.2.7, 

Coverage/Frequency  Analysis 

7.3.2. 8, 

Mutation  Analysis 

7. 3. 2. 9, 

Symbolic  Execution 

7.3.2.10, 

Regression  Testing 

7.3.2  V 

Emulation 

7.3.2 

Tuning 

7.3.2  . 

Reliability  Analysis 

7.3.2.16, 

Real  Time  Analysis 

7.3.2.17, 

Path  and  Domain  Selection 

7.3.2.18, 

Formal  Verification 

7.3.3] 
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4.6.3  Formal  Qualification  Testing 

Cross  References; 

Products: 

[Software  Test  Description 
Functions: 


4.6.4  Software  Product  Evaluations 

Cross  References: 

Products: 

Functions: 

[Source  Reconstruction 
Decompilation 
Disassembling 
Comparison 

Completeness  Checking 
Consistency  Checking 
Cross  Reference 
Maintainability  Analysis 
Scanning 
Auditing 
Sizing  Analysis 
Resource  Utilization 
Timing  Analysis 


4.6.5  Configuration  Management 

Cross  References: 

Products: 

Functions: 


(STD)] 


7.1. 7.4, 

7.1. 7.5, 

7.1. 7.6, 
7.3.1. 1, 

7.3.1.12, 

7.3.1.13, 

7.3.1.17, 

7.3.1.18, 
7.3.1.20, 
7.3.1.22, 
7.3.1.30, 
7.3.2.12, 
7.3.2.14] 
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4.6.6  Transitioning  to  Software  Support 

Cross  References: 


Products: 

[Computer  System  Operator’s  Manual  (CSOM), 

Software  User’s  Manual  (SUM), 

Software  Programmer’s  Manual  (SPM), 

Firmware  Support  Manual  (FSM)j 

Functions: 

[Predefined  and  User-Defined  Forms  7. 1.2. 3] 
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4.7  CSC  INTEGRATION  AND  TESTING 


4.7.1  Software  Development  Management 

Cross  References: 


Products: 


[Software  Development  Plan 

(SDP)] 

Functions: 

[Predefined  and  User-Defined  Forms 

7. 1.2. 3, 

Resource  Estimation 

7.2. 2. 5, 

Tracking 

7.2.2.6] 

4.7.2  Software  Engineering 

Cross  References: 

Products: 

[Software  Development  File 

(SDF)] 

Functions: 

[Data  Editing 

7.1. 1.2, 

Predefined  and  User-Defined  Forms 

7.1. 2.3, 

Assembling 

7.1. 6. 6, 

Compilation 

7.1 .6.7, 

Conversion 

7. 1.6. 8, 

Macro  Expansion 

7. 1.6. 9, 

Structure  Preprocessing 

7.1.6.10, 

Body  Stub  Generation 

7.1.6.11, 

Preamble  Generation 

7.1.6.12, 

Linking/Loading 

7.1.6.13, 

Interpretation 

7.1.6.14, 

Software  and  System  Test  Communications 

7.1.6.15, 

Program  Generation 

7.1 .7.3, 

Test  Harness  Generation 

7.1. 7.7, 

Database  Management 

7.2.1. 1, 

Documentation  Management 

7.2.1 .2, 

File  Management 

7.2.1. 3, 

Program  Library  Management 

7.2.1. 7, 

Test  Data  Management 

7.2. 1.8, 

Data  and  Error  Logging 

7.2.1.11, 

Status  Display 

7.2.1.12, 
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4.7.2  Software  Engineering  (Continued) 

Functions: 

I/O  Support 

Runtime  Environment 

Comparison 

Data  Flow  Analysis 

Interface  Analysis 

Testability  Analysis 

Test  Condition  Analysis 

Guality  Measurement 

Complexity  Measurement 

Correctness  Checking 

Completeness  Checking 

Consistency  Checking 

Reachability  Analysis 

Cross  Reference 

Maintainability  Analysis 

Auditing 

Error  Checking 

Statistical  Analysis 

Type  Analysis 

Units  Analysis 

I/O  Specification  Analysis 

Sizing  Analysis 

Data  Reduction  and  Analysis 

Random  Test  Generation 

Debugging 

Executable  Assertion  Checking 
Coverage/Frequency  Analysis 
Mutation  Analysis 
Regression  Testing 
Resource  Utilization 
Emulation 
Timing  Analysis 
Tuning 

Reliability  Analysis 
Real  Time  Analysis 
Problem  Report  Analysis 


7. 2. 3. 2, 

7. 2. 3.5, 
7.3.1. 1, 

7.3. 1.3, 

7.3.1. 5, 

7.3.1. 7, 

7.3. 1.8, 

7. 3. 1.9, 

7.3.1.10, 

7.3.1.11, 

7.3.1.12, 

7.3.1.13, 

7.3.1.16, 

7.3.1.17, 

7.3.1.18, 

7.3.1.22, 

7.3.1.23, 

7.3.1.24, 

7.3.1.27, 

7.3.1.28, 

7.3.1.29, 

7.3.1.30, 

7.3.1.31, 

7.3.1.32, 

7. 3. 2. 5, 

7. 3. 2. 6, 

7. 3. 2. 8, 

7. 3. 2. 9, 

7.3.2.11, 

7.3.2.12, 

7.3.2.13, 

7.3.2.14, 

7.3.2.15, 

7.3.2.16, 

7.3.2.17, 
7.3.4] 


4-22 


E&V  Reference  Manual,  Version  2.0 


4.7.3  Formal  Qualification  Testing 

Cross  References: 

Products: 

[Software  Test  Description 

(STD)] 

Functions: 

[Data  Editing 

7.1. 1.2, 

Predefined  and  User-Defined  Forms 

7. 1.2. 3, 

Software  and  System  Test  Communications 

7.1.6.15, 

Program  Generation 

7.1 .7.3, 

Test  Harness  Generation 

7.1. 7.7, 

Database  Management 

7.2.1. 1, 

Documentation  Management 

7. 2. 1.2, 

File  Management 

7. 2. 1.3, 

Test  Data  Management 

7.2.1. 8, 

Data  and  Error  Logging 

7.2.1.11, 

Status  Display 

7.2.1.12, 

I/O  Support 

7.2. 3. 2, 

Runtime  Environment 

7. 2. 3. 5, 

Comparison 

7.3.1. 1, 

Test  Condition  Analysis 

7.3.1. 8, 

Correctness  Checking 

7.3.1.11, 

Cross  Reference 

7.3.1.17, 

Auditing 

7.3.1.22, 

Data  Reduction  and  Analysis 

7.3.1.31, 

Random  Test  Generation 

7.3.1.32, 

Executable  Assertion  Checking 

7.3. 2. 6, 

Real  Time  Analysis 

7.3.2.17] 
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4.7.4  Software  Product  Evaluations 

Cross  References: 


Products: 

Functions: 

[Requirements  Reconstruction 

7.1. 7.2, 

Source  Reconstruction 

7.1. 7.4, 

Decompilation 

7.1. 7.5, 

Disassembling 

7.1. 7.6, 

Comparison 

7.3.1. 1, 

Completeness  Checking 

7.3.1.12, 

Consistency  Checking 

7.3.1.13, 

Cross  Reference 

7.3.1.17, 

Maintainability  Analysis 

7.3.1.18, 

Scanning 

7.3.1.20, 

Auditing 

7.3.1.22, 

Sizing  Analysis 

7.3.1.30, 

Resource  Utilization 

7.3.2.12, 

Timing  Analysis 

7.3.2.14] 

4.7.5  Configuration  Management 

Cross  References: 

Products: 

Functions: 


4.7.6  Transitioning  to  Software  Support 

Cross  References: 

Products: 

[Computer  System  Operator’s  Manual 
Software  User’s  Manual 
Software  Programmer’s  Manual 
Firmware  Support  Manual 

Functions: 

[Predefined  and  User-Defined  Forms  7. 1.2.3] 


(CSOM), 

(SUM), 

(SPM), 

(FSM)] 
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4.8  CSCI  TESTING 


4.8.1  Software  Development  Management 


Cross  References: 

Products: 

[Software  Development  Plan 
Functions: 

[Predefined  and  User-Defined  Forms 
Resource  Estimation 
Tracking 


4.8.2  Software  Engineering 


Cross  References: 


Products: 

[Software  Development  File 


Functions: 

[Data  Editing 

Predefined  and  User-Defined  Forms 

Assembling 

Compilation 

Conversion 

Macro  Expansion 

Structure  Preprocessing 

Preamble  Generation 

Linking/Loading 

Interpretation 

Software  and  System  Test  Communications 

Program  Generation 

Test  Harness  Generation 

Database  Management 

Documentation  Management 

File  Management 

Program  Library  Management 

Test  Data  Management 

Data  and  Error  Logging 

Status  Display 


(SDP)] 


7.1. 2.3. 
7. 2. 2. 5, 
7. 2. 2. 6] 


(SDF)] 


7.1. 1.2, 

7.1 .2.3, 

7.1. 6.6, 

7. 1.6. 7, 

7.1. 6.8, 

7.1 .6.9, 

7.1.6.10, 

7.1.6.12, 

7.1.6.13, 

7.1.6.14, 

7.1.6.15, 

7.1 .7.3, 

7.1. 7.7, 

7.2.1. 1, 

7.2.1 .2, 

7.2.1 .3, 

7.2.1. 7, 

7.2.1. 8, 

7.2.1.11, 

7.2.1.12, 
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4.8.2  Software  Engineering  (Continued) 

Functions: 


I/O  Support 

7.2.3. 2, 

Runtime  Environment 

7.2.3. 5, 

Import/Export 

7.2. 3. 6, 

Comparison 

7.3.1. 1, 

Data  Flow  Analysis 

7.3.1. 3, 

Interface  Analysis 

7. 3. 1.5, 

Testability  Analysis 

7.3.1. 7, 

Test  Condition  Analysis 

7. 3. 1.8, 

Quality  Measurement 

7. 3. 1.9, 

Complexity  Measurement 

7.3.1.10, 

Correctness  Checking 

7.3.1.11, 

Completeness  Checking 

7.3.1.12, 

Consistency  Checking 

7.3.1.13, 

Reachability  Analysis 

7.3.1.16, 

Cross  Reference 

7.3.1.17, 

Maintainability  Analysis 

7.3.1.18, 

Auditing 

7.3.1.22, 

Error  Checking 

7.3.1.23, 

Statistical  Analysis 

7.3.1.24, 

Type  Analysis 

7.3.1.27, 

Units  Analysis 

7.3.1.28, 

I/O  Specification  Analysis 

7.3.1.29, 

Sizing  Analysis 

7.3.1.30, 

Data  Reduction  and  Analysis 

7.3.1.31, 

Random  Test  Generation 

7.3.1.32, 

Debugging 

7. 3. 2. 5, 

Executable  Assertion  Checking 

7. 3. 2. 6, 

Coverage/Frequency  Analysis 

7. 3. 2. 8, 

Mutation  Analysis 

7. 3. 2. 9, 

Regression  Testing 

7.3.2.11, 

Resource  Utilization 

7.3.2.12, 

Emulation 

7.3.2.13, 

Timing  Analysis 

7.3.2.14, 

Tuning 

7.3.2.15. 

Reliability  Analysis 

7.3.2.16, 

Real  Time  Analysis 

7.3.2.17, 

Problem  Report  Analysis 

7.3.4] 
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4.8.3  Formal  Qualification  Testing 

Cross  References: 

Products: 

[Software  Test  Description 
Software  Test  Report 

Functions: 

[Predefined  and  User-Defined  Forms 
Software  and  System  Test  Communications 
Program  Generation 
Test  Harness  Generation 
Database  Management 
Documentation  Management 
File  Management 
Test  Data  Management 
Data  and  Error  Logging 
Status  Display 
I/O  Support 
Runtime  Environment 
Comparison 
Correctness  Checking 
Completeness  Checking 
Statistical  Analysis 
Sizing  Analysis 
Data  Reduction  and  Analysis 
Random  Test  Generation 
Executable  Assertion  Checking 
Resource  Utilization 
Timing  Analysis 


4.8.4  Software  Product  Evaluations 
Cross  References: 

Products: 

Functions: 

[Completeness  Checking 
Consistency  Checking 
Cross  Reference 
Maintainability  Analysis 
Auditing 
Sizing  Analysis 
Resource  Utilization 
Timing  Analysis 


(STD). 

(STR)] 


7.1. 2.3, 
7.1.6.15, 

7.1. 7.3, 

7.1. 7.7, 

7.2.1. 1, 

7.2.1. 2, 

7. 2. 1.3, 

7.2.1. 8. 

7.2.1.11, 

7.2.1.12, 
7. 2. 3. 2, 

7. 2. 3. 5, 
7.3.1. 1, 

7.3.1.11, 

7.3.1.12, 
7.3.1.24, 

7.3.1.30, 

7.3.1.31, 

7.3.1.32, 

7. 3. 2. 6, 

7.3.2.12, 
7.3.2.14] 


7.3.1.12, 

7.3.1.13, 

7.3.1.17, 

7.3.1.18, 
7.3.1.22, 
7.3.1.30, 
7.3.2.12, 
7.3.2.14] 
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4.8.5  Configuration  Management 

Cross  References: 

Products: 

[Version  Description  Document 
Functions: 

[Predefined  and  User-Defined  Forms 


4.8.6  Transitioning  to  Software  Support 

Cross  References: 

Products: 

[Computer  System  Operator’s  Manual 
Software  User’s  Manual 
Software  Programmer’s  Manual 
Firmware  Support  Manual 

Functions: 

[Predefined  and  User-Defined  Forms 
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(VDD)] 


7.1. 2.3] 


(CSOM). 

(SUM). 

(SPM), 

(FSM)] 


7.1. 2.3] 
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4.9  SYSTEM  INTEGRATION  AND  TESTING 


4.9.1  Software  Development  Management 

Cross  References: 

Products: 

Functions: 


4.9.2  Software  Engineering 


Cross  References: 


Products: 

[Software  Development  File 


Functions: 

[Data  Editing 

Predefined  and  User-Defined  Forms 

Assembling 

Compilation 

Conversion 

Macro  Expansion 

Structure  Preprocessing 

Preamble  Generation 

Linking/Loading 

Interpretation 

Software  and  System  Test  Communications 

Program  Generation 

Test  Harness  Generation 

Database  Management 

Documentation  Management 

File  Management 

Program  Library  Management 

Test  Data  Management 

Data  and  Error  Logging 

Status  Display 

I/O  Support 

Runtime  Environment 


(SDF)] 


7.1. 1.2, 

7.1. 2.3, 

7. 1.6. 6, 

7.1 .6.7, 

7. 1.6. 8, 

7.1. 6. 9, 

7.1.6.10, 

7.1.6.12, 

7.1.6.13, 

7.1.6.14, 

7.1.6.15, 

7.1 .7.3, 

7.1. 7.7, 

7.2.1. 1, 

7.2.1. 2, 

7.2.1. 3, 

7.2.1 .7, 

7.2.1 .8, 

7.2.1.11, 

7.2.1.12, 
7. 2. 3. 2, 
7. 2. 3. 5, 
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4.9.2  Software  Engineering  (Continued) 


Functions: 


Import/Export 

7. 2. 3. 6, 

Comparison 

7.3.1. 1. 

Data  Flow  Analysis 

7. 3. 1.3, 

Interface  Analysis 

7. 3. 1.5, 

Testability  Analysis 

7.3. 1.7, 

Test  Condition  Analysis 

7.3.1. 8, 

Quality  Measurement 

7.3.1 .9, 

Complexity  Measurement 

7.3.1.10, 

Correctness  Checking 

7.3.1.11, 

Completeness  Checking 

7.3.1.12, 

Consistency  Checking 

7.3.1.13, 

Reachability  Analysis 

7.3.1.16, 

Cross  Reference 

7.3.1.17, 

Maintainability  Analysis 

7.3.1.18, 

Auditing 

7.3.1.22, 

Error  Checking 

7.3.1.23, 

Statistical  Analysis 

7.3.1.24, 

Type  Analysis 

7.3.1.27, 

Units  Analysis 

7.3.1.28, 

I/O  Specification  Analysis 

7.3.1.29, 

Sizing  Analysis 

7.3.1.30, 

Data  Reduction  and  Analysis 

7.3.1.31, 

Random  Test  Generation 

7.3.1.32, 

Debugging 

7. 3. 2. 5, 

Executable  Assertion  Checking 

7. 3. 2. 6, 

Coverage/Frequency  Analysis 

7. 3. 2. 8, 

Mutation  Analysis 

7. 3. 2. 9, 

Regression  Testing 

7.3.2.11, 

Resource  Utilization 

7.3.2.12, 

Emulation 

7.3.2.13, 

Timing  Analysis 

7.3.2.14, 

Tuning 

7.3.2.15, 

Reliability  Analysis 

7.3.2.16, 

Real  Time  Analysis 

7.3.2.17, 

Problem  Report  Analysis 

7.3.4] 
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4.9.3  Formal  Qualification  Testing 

Cross  References: 

Products: 


[Software  Test  Description 

(STD), 

Software  Test  Report 

(STR)] 

Functions: 

[Software  and  System  Test  Communications 

7.1.6.15, 

Data  and  Error  Logging 

7.2.1.11, 

Runtime  Environment 

7. 2. 3. 5, 

Statistical  Profiling 

7.3.1.25, 

Sizing  Analysis 

7.3.1.30, 

Data  Reduction  and  Analysis 

7.3.1.31, 

Timing  Analysis 

7.3.2.14] 

4.9.4  Software  Product  Evaluations 

Cross  References: 

Products: 

Functions: 

[Completeness  Checking 

7.3.1.12, 

Consistency  Checking 

7.3.1.13, 

Cross  Reference 

7.3.1.17, 

Maintainability  Analysis 

7.3.1.18, 

Auditing 

7.3.1.22, 

Sizing  Analysis 

7.3.1.30, 

Resource  Utilization 

7.3.2.12, 

Timing  Analysis 

7.3.2.14] 
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4.9.5  Configuration  Management 

Cross  References: 

Products: 

[Version  Description  Document  (VDD)] 

Functions: 

[Predefined  and  User-Defined  Forms  7. 1.2.3] 

4.9.6  Transitioning  to  System  Support 

Cross  References: 

Products: 

Functions: 
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4.10  OPERATIONAL  TESTING  AND  EVALUATION 

4.10.1  Software  Development  Management 

Cross  References: 

Products: 

Functions: 


4.10.2  Software  Engineering 

Cross  References: 


Products: 


[Software  Development  File 

(SDF)] 

notions: 

[Data  Editing 

7.1. 1.2, 

Predefined  and  User-Defined  Forms 

7.1. 2.3, 

Assembling 

7.1. 6.6, 

Compilation 

7.1. 6.7, 

Conversion 

7.1. 6.8, 

Macro  Expansion 

7.1 .6.9, 

Structure  Preprocessing 

7.1.6.10, 

Preamble  Generation 

7.1.6.12, 

Unking/Loading 

7.1.6.13, 

Interpretation 

7.1.6.14, 

Software  and  System  Test  Communications 

7.1.6.15, 

Program  Generation 

7.1. 7.3, 

Test  Harness  Generation 

7.1. 7.7, 

Database  Management 

7.2. 1.1, 

Documentation  Management 

7. 2. 1.2, 

File  Management 

7.2.1. 3, 

Program  Library  Management 

7.2.1. 7, 

Test  Data  Management 

7.2.1. 8, 

Data  and  Error  Logging 

7.2.1.11, 

Status  Display 

7.2.1.12, 

I/O  Support 

7.2.3. 2. 

Runtime  Environment 

7. 2.3. 5. 
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4.10.2  Software  Engineering  (Continued) 

Functions: 


Import/Export 

7. 2. 3. 6, 

Comparison 

7.3.1. 1. 

Data  Flow  Analysis 

7.3.1 .3, 

Interface  Analysis 

7.3.1. 5, 

Testability  Analysis 

7.3.1. 7, 

Test  Condition  Analysis 

7.3.1. 8. 

Quality  Measurement 

7.3.1 .9, 

Complexity  Measurement 

7.3.1.10, 

Correctness  Checking 

7.3.1.11, 

Completeness  Checking 

7.3.1.12, 

Consistency  Checking 

7.3.1.13, 

Reachability  Analysis 

7.3.1.16, 

Cross  Reference 

7.3.1.17, 

Maintainability  Analysis 

7.3.1.18, 

Auditing 

7.3.1.22, 

Error  Checking 

7.3.1.23, 

Statistical  Analysis 

7.3.1.24, 

Type  Analysis 

7.3.1.27, 

Units  Analysis 

7.3.1.28, 

I/O  Specification  Analysis 

7.3.1.29, 

Sizing  Analysis 

7.3.1.30, 

Data  Reduction  and  Analysis 

7.3.1.31, 

Random  Test  Generation 

7.3.1.32, 

Debugging 

7. 3. 2. 5, 

Executable  Assertion  Checking 

7. 3. 2. 6, 

Coverage/Frequency  Analysis 

7. 3. 2. 8, 

Mutation  Analysis 

7. 3. 2. 9, 

Regression  Testing 

7.3.2.11, 

Resource  Utilization 

7.3.2.12, 

Emulation 

7.3.2.13, 

Timing  Analysis 

7.3.2.14, 

Tuning 

7.3.2.15, 

Reliability  Analysis 

7.3.2.16, 

Real  Time  Analysis 

7.3.2.17, 

Problem  Report  Analysis 

7.3.4] 
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4.10.3  Formal  Qualification  Testing 

Cross  References: 

Products: 


[Software  Test  Description 

(STD), 

Software  Test  Report 

(STR)] 

Functions: 

[Software  and  System  Test  Communications 

7.1.6.15, 

Data  and  Error  Logging 

7.2.1.11, 

Runtime  Environment 

7. 2. 3. 5, 

Statistical  Profiling 

7.3.1.25, 

Sizing  Analysis 

7.3.1.30, 

Data  Reduction  and  Analysis 

7.3.1.31. 

Resource  Utilization 

7.3.2.12, 

Timing  Analysis 

7.3.2.14] 

4.10.4  Software  Product  Evaluations 

Cross  References: 

F.cducts: 

Functions: 

[Completeness  Checking 

7.3.1.12, 

Consistency  Checking 

7.3.1.13, 

Cross  Reference 

7.3.1.17, 

Maintainability  Analysis 

7.3.1.18, 

Auditing 

7.3.1.22, 

Sizing  Analysis 

7.3.1.30, 

Resource  Utilization 

7.3.2.12, 

Timing  Analysis 

7.3.2.14] 
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4.10.5  Configuration  Management 

Cross  References: 

Products: 

Functions: 

[Evaluation  Results  Management  7.7.  1.9] 

4.10.6  Transitioning  to  System  Support 

Cross  References: 

Products: 

Functions: 


I 
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4.11  CHANGE  REQUIREMENTS 

4.11.1  System  Development  Management 

Cross  References: 

Products: 

Functions: 


4.11.2  System  Engineering 

Cross  References: 

Products: 

[Engineering  Change  Proposal 
Functions: 

[Predefined  and  User-Defined  Forms 
Change  Impact  Analysis 


4.11.3  Formal  Qualification  Testing 

Cross  References: 

Products: 


Functions: 


(ECP)] 

7. 1.2. 3, 
7. 3. 5.1] 
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4.11.4  System  Product  Evaluations 

Cross  References: 

Products: 

Functions: 

4.11.5  Configuration  Management 

Cross  References: 

Products: 

[Specification  Change  Notice 
Functions: 

[Predefined  and  User-Defined  Forms 

4.11.6  Transitioning  to  System  Support 

Cross  References: 

Products: 

Functions: 


(SCN)] 

7.1. 2.3] 
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4.12  GLOBAL 


4.12.1  System  Development  Management 
Cross  References: 

Products: 


Functions- 


[Encryption 

7.1.6.18, 

Decryption 

7.1.6.19, 

Database  (Object)  Management 

7.2.1. 1, 

Documentation  Management 

7.2.1. 2, 

File  Management 

7. 2. 1.3, 

Electronic  Mail 

7.2.1. 4, 

Electronic  Conferencing 

7.2.1. 5. 

Performance  Monitoring 

7.2.1.10, 

Cost  Estimation 

7.2.2. 1 , 

Duality  Specification 

7. 2.2.2, 

Scheduling 

7. 2. 2. 3, 

Work  Breakdown  Structure 

7. 2. 2. 4, 

Risk  Analysis 

7. 2. 2. 9, 

Access  Control 

7.2.3. 7] 

4.12.2  System  Engineering 

Cross  References: 

Products: 

Functions: 

[Text  Editing 

7.1. 1.1, 

Graphics  Editing 

7.1. 1.3, 

MIL-STD  Format 

7.1 .2.1 , 

Table  of  Contents 

7.1. 2.2, 

On-Line  Assistance  Processing 

7.1.3, 

Sort/Merge 

7.1.4, 

Graphics  Generation 

7.1.5, 

Compression 

7.1.6.16, 

Expansion 

7.1.6.17, 

Command  Language  Processing 

7.2.3. 1 , 

Input/Output  Support 

7. 2. 3. 2, 

Kernel 

7.2. 3. 3, 

Math/Statistics 

7. 2. 3. 4, 

Import/Export 

7. 2. 3. 6, 

Job  Scheduling 

7. 2. 3. 8, 

Resource  Management 

7. 2. 3. 9] 
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4.12.3  Formal  Qualification  Testing 

Cross  References: 

Products: 

Functions: 


4.12.4  System  Product  Evaluations 

Cross  References: 

Products: 


Functions: 
[Comparison 
Spelling  Checking 


4.12.5  Configuration  Management 

C/0£S-3efe.rences: 

Products: 

Functions: 

[Configuration  Management 


4.12.6  Transitioning  to  System  Support 

Cross  References: 

Products: 

Functions: 


7.3.1. 1, 
7.3.1 .2] 


7.2.2.7J 
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5. 


APSE  TOOL  CATEGORIES 


This  chapter  deals  with  APSEs,  toolsets,  and  tools.  E&V  technology  is  always 
applied  to  APSEs  or  their  components  and  thus  this  index  will  be  a  natural  starting  point  for  many 
users  of  the  RM.  Additional  discussions  of  APSE  definitions  and  views  are  provided  in 
Sections  3.1  and  3.2,  respectively.  The  following  sections  describe  the  APSE  and  its  com¬ 
ponents  and  relate  the  components  to  functions  as  shown  in  Fig.  5-1 .  The  relationships  between 
tools  and  functions  given  in  this  chapter  are  typical  for  traditional!  relationships.  The  real 
capabilities  should  be  determined  bv  the  tool  specifications,  marketing  claims,  or  the  like. 

There  is  also  a  relationship  between  tools  and  attributes.  Some  attributes  are 
related  to  qualities  of  the  function  (s)  performed  (such  as  completeness)  by  the  software,  while 
other  attributes  relate  to  non-functional  aspects  (such  as  modularity)  of  the  software.  Thus,  to 
determine  how  to  assess  a  tool,  both  the  tool-function  relationships  and  the  attributes  must  be 
explored  to  identify  the  metrics  to  be  used. 

APSEs  are  the  highest  level  of  toolset.  APSEs  should  contain  all  the  tools  and 
toolsets  needed  to  support  the  full  spectrum  of  project  activities.  APSEs.  like  toolsets,  have 
characteristics  that  mav  be  assessed  as  a  whole  fWhole  APSE  Assessment  Issues  3) .  Toolsets 
are  one  or  more  tools  which  are  intimately  coupled  and  support  a  related  set  of  functions  within 
the  APSE.  Each  section  in  this  chapter  describes  a  toolset  category  that  might  be  found  in  an 
APSE.  The  section  titles  are: 

G-I6r6 

8-11-89 


6.  ATTRIBUTES 

m 

5.  TOOLS 

■ 

7.  FUNCTIONS  b 

m 

■ 

1 

n 

Figure  5-1  Tool  Re'ationships 
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•  Computer  Management  System 

•  Information  Management  System 

•  Tool  Support  Components 

•  Distributed  APSE  Support 

•  Project  Management  System 

•  Desktop  System 

•  Configuration  Management  System 

•  Document  Generation  System 

•  Requirements/Design  Support 

•  Distributed  System  Development  and  Runtime  Support 

•  Implementation  Support 

•  Compilation  System 

•  Target  Code  Generation  and  Analysis  System 

•  Test  System. 

The  subsections  within  each  section  give  the  tool  categories  that  comprise  the  toolset.  The 
complete,  two-level  tool  taxonomy  may  be  seen  by  examining  the  Table  of  Contents  under 
Chapter  5.  Not  all  toolsets  are  necessary  for  an  APSE;  likewise,  for  a  specific  toolset,  all  tools 
may  not  always  be  present. 
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i 

1  5.1  COMPUTER  MANAGEMENT  SYSTEM 

I 

Description: 

|  Those  tools  needed  to  access,  use,  and  maintain  the  hardware  and  software  which 

consists  of  the  APSE  and  the  system  on  which  it  runs. 


5.1.1  Command  Language  Interface 

Cross  References: 

Functions: 

[Command  Language  Processing  7.2.3. 1] 


5.1.2  Archive,  Backup,  and  Retrieval  System 


Cross  References: 


Functions: 

[Compression 

Expansion 

Import/Export 


7.1.6.16, 

7.1.6.17, 
7.2.3.6] 


5.1.3  Security  System 


Cross  References; 

Functions: 

[Encryption 

Decryption 

Kernel 

Access  Control 


7.1.6.18, 

7.1.6.19, 
7. 2.3.3, 
7.2.3.7J 
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5.1.4  Job  Scheduler 


Cross  References: 

Functions: 

[Kernel  7. 2. 3. 3, 

Runtime  Environment  7. 2. 3. 5, 

Job  Scheduling  7. 2. 3. 8] 


5.1.5  Resource  Controller 


Cross  References: 

Functions: 

[Kernel  7.2.3.3. 

Runtime  Environment  7. 2. 3. 5, 

Resource  Management  7. 2. 3. 9] 


5.1.6  Import/Export  System 

Cross  References: 

Functions: 

[Import/Export  7. 2. 3. 6] 
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5.1.7  On-Line  Assistance 


Cross  References: 

Functions: 

[On-Line  Assistance  Processing  7.1.3] 


5.1.8  Input  and  Output  Services 


Cross  References: 

Functions: 

[Input/Output  Support  7. 2. 3. 2] 


5.1.9  Performance  Monitor 


Cross  References: 

Functions: 

[Performance  Monitoring  7.2.1.10] 
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5.2  INFORMATION  MANAGEMENT  SYSTEM 


Description: 

Those  tools  needed  to  control  the  information  flow  during  the  development  of  software 
systems.  This  includes  the  organization,  accession,  modification,  dissemination,  and 
processing  of  any  associated  information. 


5.2.1  File  Manager 


Cross  References: 

Functions: 

[File  Management 


7.2.1 .3] 


5.2.2  Database  Manager 


Cross  References: 

Functions: 

[Sort/Merge  7.1.4, 

Database  Management  7.2. 1.1] 
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5.3  TOOL  SUPPORT  COMPONENTS 


Description: 

Those  tools  needed  to  provide  interfaces  between  other  tools  and  host  system  services. 

5.3.1  Virtual  Operating  System 


Cross  References: 


Functions: 


[Database  (Object)  Management 

7.2.1. 1, 

File  Management 

7.2.1. 3, 

Input/Output  Support 

7. 2. 3. 2, 

Kernel 

7. 2. 3. 3, 

Runtime  Environment 

7. 2. 3. 5, 

Import/Export 

7. 2. 3. 6, 

Access  Control 

7. 2. 3. 7, 

Job  Scheduling 

7. 2. 3. 8, 

Resource  Management 

7.2. 3. 9] 

5.3.2  Window  Manager 


Cross  References! 

Functions: 

[Input/Output  Support  7. 2.3. 2, 

Kernel  7. 2. 3. 3] 


5.3.3  Language  Bindings  to  Standard  interface  Specification 
Implementations 


Cross  References: 

Functions: 
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5.3.4  I/O  Pipes 

Cross  References: 

Functions: 

[Input/Output  Support 

5.3.5  RAM  Cache 


Cross  References: 

Functions: 

[Input/Output  Support 
Emulation 


7.2.3.2J 


7. 2. 3. 2, 
7.3.2.13] 
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5.4  DISTRIBUTED  APSE  SUPPORT 


Description: 

Those  tools  needed  to  provide  the  cooperative  communication  in  supporting  the  develop¬ 
ment  of  mission  critical  software  at  diverse  geographical  locations. 
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5.5  PROJECT  MANAGEMENT  SYSTEM 


Description: 

Those  tools  needed  to  plan,  develop,  and  maintain  an  applications  system. 


5.5.1  Cost  Estimator 
Cross  References: 

Functions: 

[Cost  Estimation  7.2.2. 1] 

5.5.2  Size  Estimator 

Cross  References: 

Functions: 

[Sizing  Analysis 
Resource  Utilization 

5.5.3  Scheduler 
Cross  References; 

Functions: 

[Scheduling  7.2. 2. 3] 


7.3.1.30, 

7.3.2.12] 
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5.5.4  Work  Breakdown  Structure  Editor 
Cross  References: 

Functions: 

[Work  Breakdown  Structure 

5.5.5  Resource  Estimator 
Cross  References: 

Functions: 

[Resource  Estimation 


5.5.6  Tracking 

GLOSS-References; 


Functions: 

[Tracking 


5.5.7  Problem  Report  Analyzer 

Ctoss-Reterences; 


Functions: 

[Problem  Report  Analysis 


7.2.2.4] 


7.2.2.5] 


7.2.2.6J 


7.3.4) 
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5.5.8  Change  Impact  Analyzer 


Functions: 

[Change  Impact  Analysis 


7.3. 5.1] 


5.5.9  Analysis  and  Reporting 


Functions: 
[Tracking 
Risk  Ans'ysis 


7. 2. 2. 6, 
7.2.2.9] 
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5.6  DESKTOP  SYSTEM 


Description: 

Those  tools  needed  to  do  the  every-day,  administrative  tasks  of  business. 


5.6.1  Spreadsheet 


Cross  References: 


Functions: 

[Predefined  and  User-Defined  Forms  7. 1.2. 3, 

Math/Statistics  7.2.3.4] 


5.6.2  Calculator 
Cross  References: 

Functions: 

[Math/Statistics  7.2.3.4] 


5.6.3  Address/Phone  Book 

Cross  References; 


Functions: 

[Sort/Merge  7.1.4, 

Electronic  Mail  7. 2. 1.4, 

Electronic  Conferencing  7.2. 1.5] 
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5.6.4  Electronic  Mail 


Cross  References: 


Functions: 
[Sort/Merge 
Electronic  Mail 


5.6.5  Electronic  Conferencing 
Cross  References: 

Functions: 

[Electronic  Conferencing 


5.6.6  Calendar 

Cross  References: 

Functions: 

[Scheduling 


5.6.7  Dictionary/Thesaurus 

Cross  References; 


Functions: 

[Sort/Merge 
Comparison 
Spelling  Checking 


7.1.4, 
7.2.1. 4] 


7.2.1. 5] 


7. 2. 2. 3] 


7.1.4, 
7.3.1. 1, 
7. 3. 1.2] 
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5.7  CONFIGURATION  MANAGEMENT  SYSTEM 


Description; 

Those  tools  needed  to  control  the  contents  of  software  systems.  This  includes  monitoring 
the  status,  preserving  the  integrity  of  released  and  developed  versions,  and  controlling  the 
effects  of  changes  throughout  the  lifetime  of  the  software  system. 


5.7.1  Configuration  Identification 


Cross  References: 

Functions: 

[Configuration  Management  7.2.2.7] 

5.7.2  Configuration  Control 

Cross  References: 

Functions: 

[Program  Library  Management 
Configuration  Management 


7.2.1. 7, 
72.2.7] 


5.7.3  Configuration  Status  Accounting 


Cross  References: 

Functions: 

[Tracking  7.2. 2.6, 

Configuration  Management  7.2. 2. 7] 
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5.7.4  Version  Control 


Cross  References: 

Functions: 

[Program  Library  Management 
Configuration  Management 


5.7.5  History 

Cross  References: 

Functions: 

[Tracking 

Configuration  Management 
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5.8  DOCUMENT  GENERATION  SYSTEM 


Description: 

Those  tools  needed  to  develop  and  produce  documents. 


5.8.1  Document  Manager 


Cross  References: 


Functions: 

[Document  Management 


7.2.1. 2] 


5.8.2  Word  Processor 
Cross  References: 


Functions: 

[Text  Editing  7. 1.1.1. 

Table  of  Contents  7. 1.2. 2, 

Predefined  and  User-Defined  Forms  7. 1.2. 3. 

Syntax  And  Semantics  Checking  7.3.1.15] 


5.8.3  Spelling  Checker 
Cross  References: 

Functions: 

[Spelling  Checking  7. 3. 1.2] 
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5.8.4  Graphics  Generator 


Cross  References: 


Functions: 

[Graphics  Editing 
Graphics  Generation 


5.8.5  Formatter 


Cross  References: 


Functions: 

[MIL-STD  Format 
Table  of  Contents 
Predefined  and  User-Defined  Forms 
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5.9  REQUIREMENTS/DESIGN  SUPPORT 


Description: 

Those  tools  needed  for  the  definition,  specification,  and  design  of  software. 


5.9.1  Definition  Language  Processor 


Cross  References: 

Functions: 

[Text  Editing  7. 1.1.1, 

System  Requirements  7. 1.6.1, 

Software  Requirements  7.1. 6. 2] 


5.9.2  Specification  Language  Processor 


Cross  References: 


Functions: 


[Text  Editing 

7.1. 1.1, 

Preliminary  Design 

7.1. 6.4. 

Detailed  Design 

7. 1.6. 5, 

Design  Generation 

7. 1.7.1, 

Specification  Management 

7.2.1 .6, 

Requirements  Simulation 

7.3.2. 1] 

5.9.3  Enterprise  Model 

Cross  References; 


Functions: 


[Graphics  Editing 

7.1. 1.3, 

Graphics  Generation 

7.1.5, 

System  Requirements 

7. 1.6.1, 

Database  Management 

7.2.1. 1, 

Simulation  and  Modeling 

7. 3. 2. 3] 
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5.9.4  Data  Modei/Dictionary 
Cross  References: 


Functions: 


[Text  Editing 

7.1. 1.1. 

Graphics  Editing 

7.1. 1.3, 

Graphics  Generation 

7.1.5, 

Software  Requirements 

7.1. 6.2, 

Preliminary  Design 

7.1 .6.4, 

Detailed  Design 

7.1. 6.5, 

Design  Generation 

7.1. 7.1, 

Database  Management 

7.2.1. 1, 

Specification  Management 

7. 2. 1.6, 

Traceability  Analysis 

7.3.1. 6. 

Correctness  Checking 

7.3.1.11, 

Completeness  Checking 

7.3.1.12, 

Consistency  Checking 

7.3.1.13, 

Cross  Reference 

7.3.1.17, 

Type  Analysis 

7.3.1.27, 

Simulation  and  Modeling 

7.3.2.3] 

5.9.5  Process  Model 

Cross  References: 

Functions: 

[Graphics  Editing 

7.1. 1.3, 

Graphics  Generation 

7.1.5, 

System  Requirements 

7.1. 6.1, 

Software  Requirements 

7.1. 6.2, 

Preliminary  Design 

7.1. 6.4, 

Detailed  Design 

7.1. 6.5, 

Design  Generation 

7.1 .7.1, 

Database  Management 

7.2.1. 1, 

Specification  Management 

7.2.1 .6, 

Interface  Analysis 

7.3.1. 5, 

Traceability  Analysis 

7.3.1 .6, 

Correctness  Checking 

7.3.1.11, 

Completeness  Checking 

7.3.1.12, 

Consistency  Checking 

7.3.1.13, 

Reusability  Analysis 

7.3.1.14, 

Cross  Reference 

7.3.1.17, 

Invocation  Analysis 

7.3.1.19, 

Simulation  and  Modeling 

7.3.2.31 
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5.9.6  Simulators 


Cross  References: 

Functions: 

[Graphics  Generation 
Requirements  Simulation 
Simulation  and  Modeling 


5.9.7  Prototyping  Tools 


Cross  References: 

Functions: 

[Text  Editing 
Graphics  Editing 
Graphics  Generation 
Preliminary  Design 
Design  Generation 
Program  Generation 
Requirements  Prototyping 
Simulation  and  Modeling 
Design  Prototyping 
Emulation 


5.9.8  Design  Library  Manager 

Cross  References; 

Functions: 

[Requirements  Reconstruction 
Database  Management 
Program  Library  Management 
Quality  Assessment 
Interface  Analysis 
Reusability  Analysis 


Version  2.0 


7.1.5, 
7.3.2. 1 , 
7. 3. 2. 3] 


7.1. 1.1, 

7.1. 1.3, 
7.1.5. 

7.1. 6.4, 

7.1. 7.1, 
7.1. 7.3, 

7. 3.2.2, 

7. 3. 2. 3, 

7. 3. 2. 4, 
7.3.2.13] 


7.1. 7.2, 
7.2.1. 1, 

7. 2. 1.7, 

7. 2. 2. 8, 
7. 3. 1.5, 
7.3.1.14] 
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5.10  DISTRIBUTED  SYSTEMS  DEVELOPMENT  AND  RUNTIME  SUPPORT 


Those  tools  needed  to  support  software  development  for  distributed  processing 
systems,  and  to  provide  runtime  support  for  distributed  processing  systems. 
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5.11  IMPLEMENTATION  SUPPORT 


Description: 

Those  tools  needed  to  construct  the  source  code  for  software.  The  source  code  being 
constructed  could  be  either  for  an  applications  program  or  for  another  tool  in  the  APSE 
(meta-tool). 


5.11.1  Applications  (Code)  Generator 
Cross  References: 


Functions: 


[Predefined  and  User-Defined  Forms 

7.1. 2.3. 

Macro  Expansion 

7.1. 6.9, 

Body  Stub  Generation 

7.1.6.11, 

Preamble  Generation 

7.1.6.12, 

Program  Generation 

7.1 .7.3, 

Interface  Analysis 

7.3.1. 5. 

Invocation  Analysis 

7.3.1.19] 

5.11.2  Database  Schema  Generator 


Cross  References: 

Functions: 

[Predefined  and  User-Defined  Forms  7. 1.2. 3, 

Structure  Preprocessing  7.1.6.10] 


5.11.3  Report  Generator 

Cross.  References; 

Functions: 

[Predefined  and  User-Defined  Forms  7. 1.2. 3. 

Program  Generation  7.1. 7.3] 
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5.11.4  Screen  Generator 

Cross  References: 

Functions: 

[Predefined  and  User-Defined  Forms  7. 1.2.3, 

Graphics  Generation  7.1.5, 

Program  Generation  7. 1.7. 3] 

5.11.5  Syntax-Directed  (Language-Sensitive)  Editor 

Cross  References: 

Functions: 

[Text  Editing  7. 1.1.1, 

Predefined  and  User-Defined  Forms  7. 1.2. 3, 

Macro  Expansion  7. 1.6.9, 

Syntax  and  Semantics  Checking  7.3.1.15] 

5.11.6  Data  Definition  Language  Processor 

Cross  References: 

Functions: 

[Database  Management  7.2. 1.1] 

5.11.7  Data  Manipulation  Language  Processor 

Cross  References: 

Functions: 

[Database  Management  7.2.1 
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5.11.8  Reusable  Components  Library 


Cross  References: 


Functions: 


[Requirements  Reconstruction 

7.1 .7.2, 

Database  Management 

7.2.1. 1, 

Program  Library  Management 

7. 2. 1.7, 

Quality  Assessment 

7. 2. 2. 8, 

Interface  Analysis 

7.3. 1.5, 

Reusability  Analysis 

7.3.1.14] 
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5.12  COMPILATION  SYSTEM 


Description: 

For  the  purposes  of  this  document,  the  compilation  system  is  defined  as  those  APSE 
components  which  are  Ada-specific:  the  compiler,  the  code  generator,  the  program  library 
management  system,  and  the  runtime  system.  All  components  required  for  validation  are 
found  here. 


5.12.1  Translating  Editor 


Cross  References: 

Functions: 

[Text  Editing  7. 1.1.1, 

Compilation  7. 1.6.7, 

Syntax  And  Semantics  Checking  7.3.1.15] 


5.12.2  Compiler  (Front  End) 


Cross  References: 

Functions: 

[Compilation 

Syntax  And  Semantics  Checking 


7.1. 6.7, 
7.3.1.15] 


5.12.3  Compiler  (Code  Generator  —  Back  End) 

Cross  References; 

Functions: 

[Compilation  7. 1.6. 7] 
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5.12.4  Interpreter 

Cross  References: 

Functions: 

[Interpretation 


5.12.5  Program  Library  Manager 


Cross  References: 

Functions: 

[Program  Library  Management 


5.12.6  Runtime  System 


Cross  References: 

Functions: 

[Input/Output  Support 
Runtime  Environment 


7.1.6.14] 


7.2.1. 7] 


7. 2. 3. 2. 
7.2. 3. 5] 
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5.13  TARGET  CODE  GENERATION  AIDS  AND  ANALYSIS  TOOLSET 


Description: 

Those  tools  needed  to  provide  visibility  into  target  processes  during  program  execution.  In 
addition,  the  tools  are  used  in  the  case  where  the  host  computer  is  also  the  target  computer. 


5.13.1  Host-Target  System  Cross-Assembler 


Cross  References: 

Functions: 

[Assembler  7. 1.6. 6, 

Conversion  7. 1.6. 8] 


5.13.2  Host-Based  Target  Linker 


Cross  References; 

Functions: 

[Linking/Loading  7.1.6.13] 


5.13.3  Host-Based  Target  Loader 


Cross  References; 

Functions: 

[Linking/Loading  7.1.6.13] 
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5.13.4  Host-Based  Target  System  Instruction-Level  Simulator 

Cross  References: 

Functions: 

[Simulation  and  Modeling  7.3. 2.3] 


5.13.5  Host-Based  Target  System  Instruction-Level  Emulator 

Cross  References: 

Functions: 

[Emulation  7.3.2.13] 


5.13.6  Host-Based  Target  Code  Symbolic  Debugger 
Cross.  References: 

Functions: 

[Debugging  7.3. 2. 5] 


5.13.7  Host-to-Target  Downloader 

Cross  References: 

Functions: 

[Import/Export  7. 2.3.6] 


5.13.8  Target-to-Host  Uploader 

Cross  References; 

Functions: 

[Import/Export  7.2. 3. 6] 
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5.14  TEST  SYSTEM 


Description; 

Those  tools  needed  to  support  and  facilitate  the  planning,  development,  execution, 
evaluation,  and  documentation  of  tests  of  software. 


5.14.1  Static  Analyzers 


Cross  References: 


Functions: 


[Data  Flow  Analysis 

7.3.1. 3, 

Functional  Analysis 

7.3.1. 4, 

Interface  Analysis 

7.3.1. 5. 

Traceability  Analysis 

7.3.1 .6, 

Testability  Analysis 

7.3.1. 7, 

Quality  Measurement 

7. 3. 1.9, 

Complexity  Measurement 

7.3.1.10, 

Completeness  Measurement 

7.3.1.12, 

Consistency  Checking 

7.3.1.13, 

Reusability  Analysis 

7.3.1.14, 

Syntax  and  Semantics  Checking 

7.3.1.15, 

Reachability  Analysis 

7.3.1.16, 

Cross  Reference 

7.3.1.17, 

Maintainability  Analysis 

7.3.1.18, 

Invocation  Analysis 

7.3.1.19, 

Scanning 

7.3.1.20, 

Structured  Walkthrough 

7.3.1.21, 

Auditing 

7.3.1.22, 

Structure  Checking 

7.3.1.26, 

Type  Analysis 

7.3.1.27, 

Units  Analysis 

7.3.1.28, 

Formal  Verification 

7.3.3] 
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5.14.2  Tool  Building  Services 
Cross  References: 


Functions: 

[Program  Generation  7. 1.7. 3] 


5.14.3  Test  Building  Services 
Cross  References: 


Functions: 


[Test  Condition  Analysis 

7.3.1 .8, 

I/O  Specification  Analysis 

7.3.1.29, 

Random  Test  Generation 

7.3.1.32, 

Executable  Assertion  Checking 

7. 3. 2. 6, 

Constraint  Evaluation 

7. 3. 2. 7, 

Coverage/Frequency  Analysis 

7.3.2. 8, 

Mutation  Analysis 

7. 3.2.9, 

Symbolic  Execution 

7.3.2.10, 

Path  and  Domain  Selection 

7.3.2.18] 

5.14.4  Test  Description  and  Preparation  Services 


Cross  References; 


Functions: 

[Data  Editing  7. 1.1. 2, 

Body  Stub  Generation  7.1.6.11, 

Software  and  System  Test  Communications  7.1.6.15, 

Test  Data  Management  7.2. 1.8, 

Comparison  7. 3. 1.1] 
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5.14.5  Test  Execution  Services 

Cross  References: 

Functions: 

[Test  Harness  Generation 
Data  and  Error  Logging 
Quality  Measurement 
Regression  Testing 
Emulation 


5.14.6  Test  Analysis  Services 

Cross  References: 

Functions: 

[Evaluation  Results  Management 
Status  Display 
Correctness  Checking 
Cross  Reference 
Statistical  Analysis 
Statistical  Profiling 
Sizing  Analysis 
Data  Reduction  and  Analysis 
Resource  Utilization 
Timing  Analysis 
Tuning 

Reliability  Analysis 
Real  Time  Analysis 


5.14.7  Decision  Support  Services 

Cross  References: 

Functions: 

[Error  Checking 
Problem  Report  Analysis 
Change  Impact  Analysis 


7.1 .7.7, 

7.2.1.11, 
7.3.1 .9, 

7.3.2.11, 
7.3.2.13] 


7.2. 1.9, 
7.2.1.12, 

7.3.1.11. 
7.3.1.17, 

7.3.1.24, 

7.3.1.25, 

7.3.1.30, 

7.3.1.31, 

7.3.2.12, 

7.3.2.14, 

7.3.2.15, 

7.3.2.16, 
7.3.2.17] 


7.3.1.23, 

7.3.4, 

7. 3. 5.1] 
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6. 


ATTRIBUTES 


Attributes  are  the  characteristics  of. tools  or  ‘‘whole  APSEs"  which  the  E&V  technol¬ 
ogy  user  evaluates  for  validates^  to  make  assessments  and  comparisons.  Therefore,  the  attrib¬ 
utes  are  integral  to  finding  E&V  technology.  This  manual  uses  a  hierarchy  of  software  attributes 
derived  from  one  presented  in  a  report  by  the  Rome  Air  Development  Center  [@RADC  1985]. 
The  focus  of  the  RADC  report  is  planning  and  designing  quality  into  application  software  through¬ 
out  the  software  life  cycle.  Nevertheless,  it  provides  an  excellent  framework  for  understanding 
all  the  issues  that  surround  software  quality  and  is  recommended  for  those  users  seeking  more 
details  into  this  complex  process.  The  RADC  report  is  therefore  chosen  as  a  basis  for  this  sec¬ 
tion  of  the  E&V  Reference  Manual  even  though  the  focus  here  is  on  system  and  support  software 
and  also  on  specifying  and  assessing  attributes  for  software  that  is  already  developed.  The  re¬ 
mainder  of  this  introductory  section  is  used  to  summarize  some  of  the  issues  that  must  be  dealt 
with  in  understanding  the  software  attributes.  These  are: 

•  The  attribute  hierarchy 

•  The  functional  dependence  of  attributes 

•  Guidance  in  the  selection  of  attributes 

•  Attribute  interrelationships. 

The  first  two  levels  of  the  attribute  hierarchy  are  shown  in  Table  6-1 .  The  highest 
level  of  the  hierarchy  (performance,  design,  and  adaptation)  shows  three  broad  categories  of 
acquisition  concerns  to  potential  users  with  regard  to  software.  The  next  level  shows  quality  fac¬ 
tors  which  are  user-oriented  terms,  each  representing  an  aspect  of  software  (or  tool)  quality. 
The  quality  factors  can  themselves  be  decomposed  into  various  criteria  as  shown  in  Table  6-2. 
The  criteria  are  software-oriented  terms  representing  software  characteristics.  The  degree  to 
which  these  characteristics  are  present  in  the  software  is  an  indication  of  the  degree  of  presence 
of  a  quality  factor.  The  criteria  can  support  more  than  one  of  the  software  quality  factors  at  the 
next  higher  level. 
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Table  6-1  Top  Level  Attribute  Hierarchy 


G-04769 


ACQUISITION  CONCERN 

USER  CONCERN 

QUALITY  FACTOR 

HOW  WELL  DOES  IT  UTILIZE  A  RESOURCE? 

EFFICIENCY 

HOW  SECURE  IS  IT? 

INTEGRITY 

PERFORMANCE- 

HOW  WELL  DOES  IT 

WHAT  CONFIDENCE  CAN  BE  PLACED  IN 

RELIABILITY 

FUNCTION? 

WHAT  IT  DOES? 

HOW  WELL  WILL  IT  PERFORM  UNDER 

ADVERSE  CONDITIONS? 

SURVIVABILITY 

HOW  EASY  IS  IT  TO  USE? 

USABILITY 

HOW  WELL  DOES  IT  CONFORM  TO  THE 
REQUIREMENTS? 

CORRECTNESS 

DESIGN- 

HOW  WELL  IS  IT 

HOW  EASY  IS  IT  TO  REPAIR? 

MAINTAINABILITY 

DESIGNED? 

HOW  EASY  IS  IT  TO  VERIFY  ITS 

PERFORMANCE? 

VERIFIABILITY,  TESTABILITY 

HOW  EASY  IS  IT  TO  EXPAND,  CHANGE,  OR 
UPGRADE  ITS  CAPABILITY  OR  PERFORMANCE? 

EXPANDABILITY,  FLEXIBILITY 

ADAPTATION- 

HOW  EASY  IS  IT  TO  INTERFACE  WITH 

INTEROPERABILITY 

HOW  ADAPTABLE 

ANOTHER  SYSTEM? 

IS  IT? 

HOW  EASY  IS  IT  TO  CONVERT  FOR  USE  IN 
ANOTHER  APPLICATION? 

REUSABILITY 

HOW  EASY  IS  IT  TO  TRANSPORT  TO 

ANOTHER  ENVIRONMENT? 

TRANSPORTABILITY 

The  quality  criteria  can  also  be  decomposed  into  metrics  which  are  software-ori¬ 
ented  details  of  the  software  characteristics.  These  metrics  represent  specific  questions, 
checklists,  or  other  tests  that  are  used  to  determine  the  extent  to  which  the  tool  has  that  charac¬ 
teristic.  The  metrics  must  often  be  tailored  to  the  particular  function(s)  that  the  software  per¬ 
forms.  Other  metrics  do  not  deal  with  functionality,  but  with  aspects  of  the  software  that  are 
independent  of  the  function (s)  performed. 
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Accuracy 

6.4.1 

Anomaly  management 

6.4.2 

(fault  or  enor  tolerance,  robustness) 
Autonomy 

6.4.5 

Capacity 

6.4.6 

Communication  effectiveness 

6.4.8 

§ 

Cost 

6.4.11 

m 

Distributedness 

6.4.12 

e 

Maturity 

6.4  18 

1 

Operability  (communicativeness) 

6.420 

S. 

Power 

6.421 

Processing  (execution)  effectiveness 

64.22 

Reconfigurability 

6424 

Required  configuration 

64.26 

Storage  effectiveness 

6.4  31 

System  accessibility 

6.432 

Traning 

64.36 

c 

Completeness 

6.4.9 

,g> 

in 

Consistency 

64.10 

$ 

o 

Traceability 

64.35 

Visibility  (test  availability) 

6.4.38 

Application  independence 

6.4.3 

Augmentability 

6.4.4 

Commonality  (data  and  communication) 

64.7 

Document  accessibility 

6.4.13 

.8 

Functional  overlap 

6.4.14 

5 

Functional  scope 

6.4.15 

CL 

i 

Generality 

6.4.16 

Rehostabilrty  (independence) 

6  425 

Retargetability  (independence) 

6427 

System  clarity 

64  33 

System  compatability 

6434 

Virtuality 

6.4.37 

Granularity 

64.17 

m 

Modularity 

64.19 

Proprietary  rights 

64  23 

i 

Self-descriptiveness 

6  428 

(5 

Simplicity 

6429 

Software  production  vehicle(s) 

6430 
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Transportability 
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The  RADC  report  not  only  defines  the  software  quality  factors,  criteria,  and  metrics, 
it  also  provides  guidance  for  their  selection  and  use.  In  particular,  there  is  guidance  for  selecting 
attributes  based  on: 


•  System  characteristics 

•  Complementary  factors 

•  Shared  criteria 

•  Attribute  interrelationships. 

The  desired  software  quality  factors  should  be  tailored  to  the  characteristics  of  the 
software  being  built  and  the  development  environment.  In  fact,  some  of  the  tools  selected  for  the 
development  environment  will  probably  be  based  on  the  system  characteristics.  For  example,  if 
a  tool  or  environment  is  to  have  a  long  life  cycle,  the  E&V  user  should  be  concerned  with  exp¬ 
andability  and  maintainability.  Likewise,  if  the  system  being  built  is  a  real-time  application,  the  de¬ 
veloper  should  consider  acquiring  tools  that  can  assess  the  efficiency  of  the  system. 

AnotheLconsideration  when  selecting  quality  factors  to  be  assessed  is  the  effect  of 
low  quality  levels  among  complementary  factors.  Four  factors  are  complementary  to  most  other 
factors  and  should  influence  the  selection  of  important  factors.  Table  6-3  shows  the  comple¬ 
mentary  factors: 

•  Reliability 

•  Correctness 

•  Maintainability 

•  Verifiability. 

Quality  levels  for  the  other  specified  factors  are  difficult  to  measure  accurately  when  there  are 
low  quality  levels  for  anv  of  the  four  complementary  factors.  For  example,  if  a  high  quality  level 
for  reliability  is  specified,  high  quality  levels  for  correctness  and  verifiability  should  also  be  speci¬ 
fied.  This  is  because  if  the  scores  for  reliability  are  high,  but  the  software  is  incorrect  or  difficult 
to  verify,  the  true  reliability  may  be  low  due  to  incorrect  software  or  uncertainty  in  verification. 
The  complementary  quality  factors  show  that  the  attributes  are  not  independent.  Anv  project. 
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Table  6-3  Complementary  Software  Quality  Factors 
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EFFICIENCY 

INTEGRITY 

□ 

□ 

□ 

RELIABILITY 

|| 

□ 

□ 

SURVIVABILITY 

□ 

||j 

□ 

□ 

USABILITY 

□ 

1 

□ 

□ 

□ 

CORRECTNESS 

MAINTAINABILITY 

□ 

□ 

11 

VERIFIABILITY,  TESTABILITY 

□ 

□ 

□ 

x-x'x 

gx-Xv 

EXPANDABILITY,  FLEXIBILITY 

□ 

□ 

□ 

□ 

nn 

INTEROPERABILITY 

□ 

□ 

□ 

□ 

REUSABILITY 

□ 

□ 

□ 

□ 

TRANSPORTABILITY 

□ 

□ 

□ 

□ 

ill 

★  =  DEPENDENCY 


regardless  of  the  type  of  system  or  application,  should  consider  the  complementary  factors 
the  quality  specifications. 

The  criteria  that  are  attributes  of  more  than  one  quality  factor  are  shared  enter 
For  example,  Table  6-2  shows  simplicity  is  a  criterion  for  five  of  the  factors.  The  beneficial  efft 
of  this  is  that  these  attributes  are  built  into  the  software  only  once:  likewise,  assessment  of  the 
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tied.  Some  factors  have  criteria  that  conflict  with  another  factor;  and  some  have  criteria  that  co¬ 
operate  with  another  factor.  Attribute  criteria  affecting  other  factors  are  shown  in  Table  6-4. 
Attributes  that  are  basic  criteria  of  a  factor  are  identified  with  an  “x”;  criteria  that  are  in  a  positive 
or  beneficial  relationship  with  another  factor  are  identified  with  a  and  criteria  that  are  in  a 
negative  or  adverse  relationship  with  another  factor  are  identified  with  a  For  example,  oper¬ 
ability  is  a  criterion  of  usability  and  is  shown  to  have  a  beneficial  relationship  with  maintainability. 
The  assertion  is  that  the  operability  of  usable  software  aids  in  software  maintenance,  even 
though  it  is  not  an  essential  characteristic  of  maintainability.  The  implication  is  that  the  effort  to 
maintain  software  will  be  less  if  usability  is  also  a  specified  quality.  An  example  of  a  criterion  with 
an  adverse  relationship  is  anomaly  management.  Anomaly  management  is  a  criterion  of  reliabil¬ 
ity  and  is  shown  to  have  a  conflicting  relationship  with  efficiency.  The  assertion  is  that  the  addi¬ 
tional  code  required  to  perform  anomaly  management  increases  the  runtime  and  requires 
additional  memory,  thus  decreasing  the  potential  efficiency.  This  implies  that  efficient  use  of  re¬ 
sources  will  be  more  difficult  to  achieve  if  reliability  is  also  a  specified  quality  factor.  Possible  so¬ 
lutions  to  this  conflict  include: 

•  Budget  and  schedule  to  try  to  achieve  high  goals  for  both  fac¬ 
tors 

•  Lowering  goals  for  one  or  the  other  factor,  including  decreasing 
emphasis  on  the  specific  criterion  that  conflicts  and  increasing 
emphasis  on  those  that  do  not  conflict 

•  Allocating  more  resources  to  the  host  system  (e.g.,  more  pro¬ 
cessing  power  or  more  memory)  and  possibly  decreasing  em¬ 
phasis  on  high  software  quality  goals. 

Figure  6-1  shows  how  the  attributes  are  related  to  other  elements  of  the  E&V  Refer¬ 
ence  Manual  and  to  elements  in  the  E&V  Guidebook.  The  various  attributes  that  are  useful  in  de¬ 
fining  assessment  objectives  for  APSEs  or  APSE  components  are  described  in  this  chapter. 
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Table  6-4  Beneficial  and  Adverse  Effects  of  Criteria  on 
Software  Quality  Factors 


G-161SJ 

8-10-89 


Acquisition  Concern 

Factor 

Criterion/Selection  X. 

Accuracy 

6.4.1 

Anomaly  management 

6.4.2 

(fault  or  error  tolerance,  robustness) 

Autonomy 

6.4.5 

Capacity 

6.4.6 

Communication  effectiveness 

6.4.8 

Cost 

6.4.11 

Distributedness 

6.4.12 

Maturity 

6.4.18 

Operability  (communicativeness) 

6.4.20 

Power 

6.421 

Processing  (execution)  effectiveness 

6.422 

Reconfigurability 

6.4  24 

Required  configuration 

64.26 

Storage  effectiveness 

6.4.31 

System  accessibility 

6.4.32 

Traning 

6.4.36 

Completeness 

6.49 

Consistency 

6.4.10 

Traceability 

6.4.35 

Visibility  (test  availability) 

6.4.38 

Application  independence 

6.4.3 

Augmentability 

6.44 

Commonality  (data  and  communication) 

6.47 

Document  accessibility 

6.4.13 

Functional  overlap 

6.4.14 

Functional  scope 

6.4.15 

Generality 

6.4.16 

Rehostability  (independence) 

6.4.25 

Retargetability  (independence) 

6.4.27 

System  clarity 

6.4.33 

System  compatability 

6.4.34 

Virtuality 

6.4.37 

Granularity 

6.4.17 

Modularity 

6.4,19 

Proprietary  rights 

6.4.23 

Self-descriptiveness 

6.428 

Simplicity 

6.429 

Software  production  vehicle(s) 

6.4.30 

Performance 

6.1 


Design 

6.2 


Adaptation 

6.3 
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-  Basic  Relationship  +  »  Positive  Effect  -  -  Negative  Effect  Blank  -  None  or  application  dependent 
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G-16377 


Figure  6-1  Attribute  Relationships 
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6.1  PERFORMANCE 


Description: 

Performance  quality  factors  deal  both  with  the  ability  of  the  software  to  function  and  with  error 
occurrences  that  affect  software  functioning.  Low  quality  factors  predict  poor  software  per¬ 
formance.  [@RADC  1985] 


Cross  References: 


Software  Quality  Factors: 

[Efficiency 

6.1.1, 

Integrity 

6.1.2, 

Reliability 

6.1.3, 

Survivability 

6.1.4, 

Usability 

6.1.5] 
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6.1.1  Efficiency 


Description: 


The  ratio  of  actual  utilization  of  the  system  resources  to  optimum  utilization.  [@GIT  1987] 


The  extent  to  which  a  component  fulfills  its  purpose  using  a  minimum  of  computing  re¬ 
sources.  Of  course,  many  of  the  ways  of  coding  efficiently  are  not  necessarily  efficient  in  the 
sense  of  being  cost-effective,  since  transportability,  maintainability,  etc.,  may  be  degraded 

as  a  result.  [@DACS  1979] 

Cross  References: 

Acquisition  Concern: 

[Performance 

6.1] 

Software-Oriented  Criteria: 

[Communication  Effectiveness 

6.4.8, 

Processing  Effectiveness 

6.4.22, 

Storage  Effectiveness 

Complementary  Software  Guality  Factors: 

Cooperating  Criteria: 

Conflicting  Criteria: 

6.4.31] 

[Accuracy 

6.4.1, 

Anomaly  Management 

6.4.2, 

Commonality 

6.4.7, 

Generality 

6.4.16, 

Modularity 

6.4.19, 

Operability 

6.4.20, 

Reconfigurability 

6.4.24, 

Rehostability 

6.4.25, 

Retargetability 

6.4.27, 

Self-Descriptiveness 

6.4.28, 

Simplicity 

6.4.29, 

System  Accessibility 

6.4.32, 

Virtuality 

6.4.37] 
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6.1.2  Integrity 
Description: 

The  probability  that  the  system  will  perform  without  failure  and  will  protect  the  system  and 
data  from  unauthorized  access.  [@GIT  1987]  The  extent  to  which  unauthorized  access  to  or 
modification  of  software  or  data  can  be  controlled  in  a  computer  system.  [@IEEE  1983] 

Cross  References: 


Acquisition  Concern: 

[Performance  6.1] 


Software-Oriented  Criteria: 

[System  Accessibility  6.4.32] 


Complementary  Software  Quality  Factors: 

[Reliability  6.1.3, 

Correctness  6.2.1, 

Verifiability,  Testability  6.2.3] 


Cooperating  Criteria: 


Conflicting  Criteria: 

[Distributedness  6.4.12, 

Document  Accessibility  6.4.13, 

Generality  6.4.16, 

System  Compatibility  6.4.34] 
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6.1.3  Reliability 


Description: 

The  probability  that  the  system  will  perform  as  intended  under  stated  conditions  for  a  speci¬ 
fied  period  of  time.  [@GIT  1987] 


Cross  References: 


Acquisition  Concern: 

[Performance  6.1] 


Software-Oriented  Criteria: 

[Accuracy  6.4.1, 

Anomaly  Management  6.4.2, 

Maturity  6.4.18, 

Simplicity  6.4.29] 


Complementary  Software  Quality  Factors: 

[Correctness  6.2.1, 

Verifiability,  Testability  6.2.3] 


Cooperating  Criteria: 


Conflicting  Criteria: 

[Generality  6.4.16] 
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6.1.4  Survivability 
Description: 

The  extent  to  which  the  software  will  perform  and  support  critical  functions  without  failures 
within  a  specified  time  period  when  a  portion  of  the  system  is  inoperable.  Survivability  deals 
with  the  continued  performance  of  software  (e.g.,  in  a  degraded  mode)  even  when  a  portion 
of  the  system  has  failed.  [@RADC  1985] 

Cross  References: 


Acquisition  Concern: 


[Performance 

6.1] 

Software-Oriented  Criteria: 

[Anomaly  Management 

6.4.2. 

Autonomy 

6.4.5, 

Distributedness 

6.4.12, 

Granularity 

6.4.17, 

Modularity 

6.4.19, 

Reconfigurability 

6.4.24] 

Complementary  Software  Quality  Factors: 

[Reliability 

6.1.3, 

Correctness 

6.2.1, 

Verifiability,  Testability 

6.2.3] 

Cooperating  Criteria: 

Conflicting  Criteria: 

[Generality 

6.4.16] 
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6.1.5  Usability 


Description: 

Extent  to  which  resources  required  to  acquire,  install,  learn,  operate,  prepare  input  for,  and 
interpret  output  of  a  component  are  minimized. 


Cross  References: 


Acquisition  Concern: 

[Performance 

6.1] 

Software-Oriented  Criteria: 

[Capacity 

6.4.6, 

Cost 

6.4.11, 

Granularity 

6.4.17, 

Maturity 

6.4.18, 

Operability 

6.4.20, 

Power 

6.4.21, 

Required  Configuration 

6.4.26, 

Training 

6.4.36] 

Complementary  Software  Quality  Factors: 

[Reliability 

6.1.3, 

Correctness 

6.2.1, 

Maintainability 

6.2.2, 

Verifiability.  Testability 

6.2.3] 

Cooperating  Criteria: 

[Anomaly  Management 

6.4.2, 

Document  Accessibility 

6.4.13] 

Conflicting  Criteria: 
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6.2  DESIGN 


Description: 

Design  quality  factors  deal  mainly  with  software  failure  and  correction.  Low  quality  levels 
usually  result  in  repeating  a  portion  of  the  development  process  (e.g.,  redesign,  recode, 
reverify);  hence  the  term  design.  [@RADC  1985] 


Cross  References: 


Software  Quality  Factors: 

[Correctness  6.2.1, 

Maintainability  6.2.2, 

Verifiability,  Testability  6.2.3] 
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6.2.1  Correctness 


Description: 

The  extent  to  which  the  system  conforms  to  its  specifications  and  standards.  [@GIT  1987] 
(1)  The  extent  to  which  software  is  free  from  design  defects  and  from  coding  defects;  that  is 
fault  free.  (2)  The  extent  to  which  software  meets  its  specified  requirements.  [@IEEE  1983] 

Cross  References: 


Acquisition  Concern: 
[Design 


6.2] 


Software-Oriented  Criteria: 

[Completeness  6.4.9, 

Consistency  6.4.10, 

Traceability  6.4.35] 


Complementary  Software  Quality  Factors: 


Cooperating  Criteria: 


Conflicting  Criteria: 
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6.2.2  Maintainability 
Description: 

The  ease  of  effort  for  locating  and  fixing  software  failures  within  a  sp-  ;ified  time  period. 
[@RADC  1985]  The  probability  that  the  system  can  be  restored  to  a  specified  condition  within 
a  specified  amount  of  time.  [@GIT  1987] 

Cross  References: 


Acauisition  Concern: 


[Design 

6.2] 

Software-Oriented  Criteria: 

[Consistency 

6.4.10, 

Granularity 

6.4.17, 

Modularity 

6.4.19, 

Proprietary  Rights 

6.4.23, 

Self-Descriptiveness 

6.4.28, 

Simplicity 

6.4.29, 

Software  Production  Vehicle (s) 

6.4.30, 

Visibility 

6.4.38] 

Complementary  Software  Quality  Factors: 

[Reliability 

6.1.3, 

Correctness 

6.2.1] 

Cooperating  Criteria: 

[Completeness 

6.4.9, 

Document  Accessibility 

6.4.13, 

Operability 

6.4.20, 

Reconfigurability 

6.4.24, 

Traceability 

6.4.35] 

Conflicting  Criteria: 

[Communication  Effectiveness 

6.4.8, 

Processing  Effectiveness 

6.4.22, 

Storage  Effectiveness 

6.4.31] 
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6.2.3  Verifiability,  Testability 
Description: 

The  extent  to  which  the  specified  system  operation  and  performance  determine  the  condi¬ 
tions  and  criteria  for  tests.  [@GIT  1987]  (1)  The  extent  to  which  software  facilitates  both  the 
establishment  of  test  criteria  and  the  evaluation  of  the  software  with  respect  to  those  criteria. 
(2)  The  extent  to  which  the  definition  of  requirements  facilitates  analysis  of  the  requirements 
to  establish  test  criteria.  (@IEEE  1983] 

Cross  References: 


Acquisition  Concern: 

[Design  6.2] 

Software-Oriented  Criteria: 

[Granularity  6.4.17, 

Modularity  6.4.19, 

Self-Descriptiveness  6.4.28, 

Simplicity  6.4.29, 

Visibility  6.4.38] 

Complementary  Software  Quality  Factors: 

[Reliability  6.1.3, 

Correctness  6.2.1, 

Maintainability  6.2.2] 

Cooperating  Criteria: 

[Consistency  6.4.10, 

Operability  6.4.20, 

Traceability  6.4.35] 

Conflicting  Criteria: 

[Communication  Effectiveness  6.4.8, 

Processing  Effectiveness  6.4.22, 

Storage  Effectiveness  6.4.31] 


6-18 


E&V  Reference  Manual,  Version  2.0 


6.3  ADAPTATION 


Description; 

Adaptation  quality  factors  deal  mainly  with  using  software  beyond  its  original  requirements, 
such  as  extending  or  expanding  capabilities  and  adapting  for  use  in  another  application  or 
environment.  Low  quality  levels  predict  relatively  high  costs  for  new  software  use.  [@RADC 
1985] 


Cross  References: 


Software  Quality  Factors: 

[Expandability,  Flexibility 

6.3.1, 

Interoperability 

6.3.2, 

Reusability 

6.3.3, 

Transportability 

6.3.4] 

6-19 


E&V  Reference  Manual,  Version  2.0 


6.3.1  Expandability,  Flexibility 


Description: 

The  extent  to  which  the  system  capability  or  performance  can  be  increased  by  enhancing 
current  functions  or  adding  new  functions  (expandability).  The  extent  to  which  system  pur¬ 
pose,  functions,  or  data  can  be  changed  to  satisfy  other  specified  requirements  (flexibility). 
[@GIT  1987]  The  extent  to  which  a  component  allows  new  capabilities  to  be  added  and  exist¬ 
ing  capabilities  to  be  easily  tailored  to  user  needs.  [@DACS  1979] 

Cross  References: 

Acquisition  Concern: 


[Adaptation 

6.3] 

Software-Oriented  Criteria: 

[Augmentability 

6.4.4, 

Generality 

6.4.16, 

Granularity 

6.4.17, 

Modularity 

6.4.19, 

Proprietary  Rights 

6.4.23, 

Retargetability 

6.4.27, 

Self-Descriptiveness 

6.4.28, 

Simplicity 

6.4.29, 

Software  Production  Vehicle(s) 

6.4.30, 

Virtuality 

6.4.37] 

Complementary  Software  Quality  Factors: 

[Reliability 

6.1.3, 

Correctness 

6.2.1, 

Maintainability 

6.2.2, 

Verifiability,  Testability 

6.2.3] 

Cooperating  Criteria: 

[Capacity 

6.4.6, 

Consistency 

6.4.10, 

Distributedness 

6.4.12, 

Required  Configuration 

6.4.26, 

Traceability 

6.4.35] 

Conflicting  Criteria: 

[Reconfigurability 

6.4.24] 
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6.3.2  Interoperability 
Description: 

Tiie  probability  that  two  or  more  systems  can  exchange  information  under  stated  conditions 
and  use  the  information  that  has  been  exchanged.  [@GIT  1987]  The  ability  of  APSEs  to  ex¬ 
change  database  objects  and  their  relationships  in  forms  usable  by  components  and  user 
programs  without  conversion.  Interoperability  is  measured  by  the  degree  to  which  this  ex¬ 
change  can  be  accomplished  without  conversion.  [@E&V  Requirements] 

Cross  References: 


Acquisition  Concern: 

[Adaptation 

6.3] 

Software-Oriented  Criteria: 

[Commonality 

6.4.7, 

Functional  Overlap 

6.4.14, 

Modularity 

6.4.19. 

Rehostability 

6.4.25, 

Retargetability 

6.4.27, 

System  Compatibility 

6.4.34] 

Complementary  Software  Quality  Factors: 

[Reliability 

6.1.3, 

Correctness 

6.2.1, 

Maintainability 

6.2.2, 

Verifiability,  Testability 

6.2.3] 

Cooperating  Criteria: 

[Generality 

6.4.16] 

Conflicting  Criteria: 
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6.3.3  Reusability 


Description: 

The  extent  to  which  a  component  can  be  adapted  fcr  use  in  another  application.  [<®RADC 
1985] 


Cross  References: 


Acquisition  Concern: 


[Adaptation 

6.3] 

Software-Oriented  Criteria: 

[Application  Independence 

6.4.3, 

Document  Accessibility 

6.4.13. 

Functional  Scope 

6.4.15, 

Generality 

6.4.16, 

Granularity 

6.4.17, 

Modularity 

6.4.19, 

Rehostability 

6.4.25, 

Retargetability 

6.4.27, 

Self-Descriptiveness 

6.4.28, 

Simplicity 

6.4.29, 

Software  Production  Vehicle (s) 

6.4.30, 

System  Clarity 

6.4.33] 

Complementary  Software  Quality  Factors: 

rr*»  — 1‘  i_  •••*_  . 


[Reliability 

6.1.3, 

Correctness 

6.2.1, 

Maintainability 

6.2.2, 

Verifiability,  Testability 

6.2.3] 

Cooperating  Criteria: 

[Completeness 

6.4.9, 

Consistency 

6.4.10, 

Required  Configuration 

6.4.26, 

Traceability 

6.4.35] 

Conflicting  Criteria: 

[Reconfigurability 

6.4.24] 
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6.3.4  Transportability 
Description: 

The  extent  io  which  a  component  can  be  adapted  for  use  in  another  environment  (e.g.,  differ¬ 
ent  host  or  target  hardware,  operating  system,  APSE).  [@RADC  1985]  The  ability  of  a  com¬ 
ponent  to  be  installed  on  a  different  APSE  without  change  in  functionality.  Transportability  is 
measured  in  the  degree  to  which  this  installation  can  be  accomplished  without  reprogram¬ 
ming.  [@E&V  Requirements] 

Cross  References: 


Acquisition  Concern: 

[Adaptation 

6.3] 

Software-Oriented  Criteria: 

[Modularity 

6.4.19, 

Rehostability 

6.4.25, 

Retargetability 

6.4.27. 

Self-Descriptiveness 

6.4.28, 

Software  Production  Vehicle(s) 

6.4.30] 

Complementary  Software  Quality  Factors: 


[Reliability  6.1.3, 

Correctness  6.2.1, 

Maintainability  6.2.2, 

Verifiability,  Testability  6.2.3] 

Cooperating  Criteria: 

[Application  Independence  6.4.3, 

Required  Configuration  6.4.26] 

Conflicting  Criteria: 

[Communication  Effectiveness  6.4.8, 

Processing  Effectiveness  6.4.22, 

Reconfigurability  6.4.24, 

Storage  Effectiveness  6.4.31] 
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6.4  SOFTWARE-ORIENTED  CRITERIA 


6.4.1  Accuracy 


Description: 

A  quantitative  measure  of  the  magnitude  of  error,  preferably  expressed  as  a  function  of  the 
relative  error,  a  high  value  of  this  measure  corresponding  to  a  small  error.  Error  is  defined  as 
a  discrepancy  between  a  computed,  observed,  or  measured  value  or  condition  and  the  true, 
specified,  or  theoretically  correct  value  or  condition.  [@IEEE  1983] 


Cross  References: 


Software  Quality  Factors: 
[Reliability 


6.1.3] 


Beneficial  Quality  Factors: 


Adverse  Quality  Factors: 
[Efficiency 


6.1.1] 


Guidebook  References: 

[@GB:  RADC  Software  Quality  Metric  Worksheets  99.5] 
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6.4.2  Anomaly  Management,  Fault  or  Error  Tolerance,  Robustness 


Description; 

Those  characteristics  of  software  which  provide  for  continuity  of  operations  under  and  recov¬ 
ery  from  non-nominal  conditions.  Such  conditions  include:  improper  input  data,  computation¬ 
al  failures,  hardware  faults,  device  errors,  communication  errors,  and  node  or 
communication  failures.  [@RADC  1985]  The  protection  of  a  component  from  itself,  user  er¬ 
rors,  and  system  errors.  The  ability  to  recover  and  provide  meaningful  diagnostics  in  the 
event  of  unforeseen  situations.  A  robust  routine  will  avoid  failing  for  input  values  where  the 
desired  output  is  well-defined,  but  the  intermediate  results  of  a  straightforward  implementa¬ 
tion  may  cause  the  routine  to  fail.  [@E&V  Requirements] 


Cross  References: 


Software  Quality  Factors: 

[Reliability  6.1.3, 

Survivability  6.1.4] 

Beneficial  Quality  Factors: 

[Usability  6.1.5] 

Adverse  Quality  Factors: 

[Efficiency  6.1.1] 


Guidebook  References; 

[@GB:  RADC  Software  Quality  Metric  Worksheets 
@GB:  SEI  Assessment  of  Software  Engineering  Tools 
•Compilation  7. 1.6. 7, 

@GB:  ARTEWG  Catalogue  of  Ada  Runtime 
Implementation  Dependencies 
Compilation  7. 1.6. 7, 

@GB:  Compiler  Assessment  Questionnaire 
Command  Language  Processing  7.2. 3.1, 

@GB:  Command  Language  Interpreter 
Assessment  Questionnaire 

Runtime  Environment  7.2. 3.5, 

@GB:  Runtime  Support  System  Questionnaire 

•NOTE:  Normally,  the  concern  of  evaluation  is  the  quality  of  a  tool.  However,  this 
evaluation  technique  focuses  on  the  products  of  the  tool  rather  than  the  tool  itself. 


99.5: 

99.6: 

5.10; 

5.12: 

99.4: 

5.14] 
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I 


6.4.3  Application  independence 


Description: 

Those  characteristics  of  software  which  determine  its  non-dependency  on  database  system, 
microcode,  computer  architecture,  data  structures,  and  algorithms.  [@RADC  1985] 


Cross  References: 


Software  Quality  Factors: 

[Reusability  6.3.3] 


Beneficial  Quality  Factors: 

[Transportability  6.3.4] 


Adverse  Quality  Factors: 


Guidebook  References: 

[@GB:  RADC  Software  Quality  Metric  Worksheets  99.5: 

Whole  APSE  Assessment  Issues  3., 

@GB:  APSE  Customization  Questionnaire  13.4] 
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6.4.4  Augmentability 


Description: 

Those  characteristics  of  software  which  provide  for  expansion  of  capability  for  functions  and 
data.  Metrics  include:  data  storage  expansion,  computational  extensibility,  channel  exten¬ 
sibility,  and  design  extensibility.  [@RADC  1985] 


Cross  References: 


Software  Quality  Factors: 

[Expandability,  Flexibility  6.3.1] 


Beneficial  Quality  Factors: 


Adverse  Quality  Factors: 


Guidebook  References: 


[@GB:  RADC  Software  Quality  Metric  Worksheets  99.5: 

@GB:  SEI  Assessment  of  Software  Engineering  Tools  99.6; 

Whole  APSE  Assessment  Issues  3., 

@GB:  Ada-Europe  Ada  Environment  Questionnaires  13.2; 

Systems  Requirements  7. 1.6.1, 

@GB:  NADC/SPS  CASE  Tools  Evaluation  9.4; 

Software  Requirements  7. 1.6. 2, 

@GB:  NADC/SPS  CASE  Tools  Evaluation  9.4; 

Preliminary  Design  7. 1.6. 4, 

@GB:  NADC/SPS  CASE  Tools  Evaluation  9.4: 

Compilation  7. 1.6. 7, 

@GB:  Compiler  Assessment  Questionnaire  5.12; 

Configuration  Management  7. 2. 2. 7 


.  @GB:  Configuration  Management  Assessment  Questionnaire  10.3: 
Command  Language  Processing  7.2.3. 1, 

@GB:  Command  Language  Interpreter 

Assessment  Questionnaire  99.4] 
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6.4.5  Autonomy 

Description: 

Those  characteristics  of  software  which  determine  its  non-dependency  on  interfaces  and 
functions.  Metrics  include:  interface  complexity  and  self-sufficiency.  [@RADC  1985] 

Cross  References: 

Software  Quality  Factors: 

[Survivability  6.1.4] 

Beneficial  Quality  Factors: 

Adverse  Quality  Factors: 

GuictebQQK- References: 

[@GB:  RADC  Software  Quality  Metric  Worksheets  99.5] 
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6.4.6  Capacity 


Description: 

The  upper  and  lower  limits  of  the  functions  implemented  by  a  tool.  For  example,  the  maxi¬ 
mum  and  minimum  quantities  of  input  data  for  a  tool  to  operate.  [@E&V  Requirements] 


Cross  References: 


Software  Quality  Factors: 


[Usability 

6.1.4] 

Beneficial  Quality  Factors: 

[Expandability,  Flexibility 

6.3.1] 

Adverse  Quality  Factors: 

GuictekoQkjgeferences: 

[@GB:  SEI  Assessment  of  Software  Engineering  Tools 

99.6; 

Whole  APSE  Assessment  Issues 

3., 

@GB:  APSE  Characterization 

13.1: 

Whole  APSE  Assessment  Issues 

3., 

@GB:  Ada-Europe  Ada  Environment  Questionnaires 

13.2: 

Systems  Requirements 

7. 1.6.1, 

@GB:  NADC/SPS  CASE  Tools  Evaluation 

9.4; 

Software  Requirements 

7.1 .6.2, 

@GB:  NADC/SPS  CASE  Tools  Evaluation 

9.4; 

Preliminary  Design 

7.1. 6.4, 

@GB:  NADC/SPS  CASE  Tools  Evaluation 

9.4; 

Compilation 

7.1 .6.7, 

@GB:  IDA  Benchmarks 

5.2; 

Compilation 

7.1 .6.7, 

@GB:  MITRE  Benchmark  Generator  Tool  (BGT) 

5.6; 

Compilation 

7.1 .6.7, 

@GB:  Compiler  Assessment  Questionnaire 

5.12; 

Configuration  Management 

7.2.21, 

@GB:  Configuration  Management  Assessment  Questionnaire  10.3: 
Command  Language  Processing  7. 2. 3.1, 

@GB:  Command  Language  Interpreter 

Assessment  Questionnaire  99.4] 
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6.4.7  Commonality  (Data  and  Communication) 


Description: 

Those  characteristics  of  software  which  provide  for  the  use  of  interface  standards  for  proto¬ 
cols,  routines,  and  data  representations.  [@RADC  1985]  The  set  of  assumptions  made  by 
the  component  and  made  about  the  component  by  the  remaining  components  and  the  sys¬ 
tem  in  which  it  appears.  Software  components  have  control,  data,  and  service  interfaces. 
[@DACS  1979] 


Cross  References: 


Software  Quality  Factors: 

[Interoperability  6.3.2] 

Beneficial  Quality  Factors: 

Adverse  Quality  Factors: 

[Efficiency  6.1.1] 


Guidebook  References: 


[@GB:  RADC  Software  Quality  Metric  Worksheets  99.5 

@GB:  SEI  Assessment  of  Software  Engineering  Tools  99.6 

Whole  APSE  Assessment  Issues  3., 

@GB:  Distributed  APSE  Questionnaire  12.1 

Whole  APSE  Assessment  Issues  3., 

@GB:  Ada-Europe  Ada  Environment  Questionnaires  13.2 

Whole  APSE  Assessment  Issues  3., 

@GB:  APSE  Customization  Questionnaire  13.4 

Compilation  7. 1.6. 7, 

@GB:  Compiler  Assessment  Questionnaire  5.12 

Configuration  Management  7.2.2. 7, 


@GB:  Configuration  Management  Assessment  Questionnaire  10.3: 
Command  Language  Processing  7.2.3. 1, 

@GB:  Command  Language  Interpreter 

Assessment  Questionnaire  99.4] 
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6.4.8  Communication  Effectiveness 


Description: 

Those  characteristics  of  the  software  which  provide  for  minimum  utilization  of  communica¬ 
tions  resources  in  performing  functions.  [@RADC  1985] 


Cross  References: 


Software  Quality  Factors: 

[Efficiency  6.1.1] 


Beneficial  Quality  Factors: 


Adverse  Quality  Factors: 

[Maintainability  6.2.2, 

Verifiability,  Testability  6.2.3, 

Transportability  6.3.4] 


Guidebook  References: 

[@GB:  RADC  Software  Quality  Metric  Worksheets  99.5; 

Runtime  Environment  7. 2. 3. 5, 

@GB:  Runtime  Support  System  Questionnaire  5.14] 
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6.4.9  Completeness 


Description: 

The  extent  to  which  a  component  provides  the  complete  set  of  operations  necessary  to  per¬ 
form  a  function.  For  example:  are  all  the  inputs  and  outputs  clearly  defined  for  each  opera¬ 
tion:  does  each  data  element  have  a  clearly  defined  meaning,  format,  and  source:  and  are  all 
operations  available  and  working  as  expected?  [@E&V  Requirements] 


Cross  References: 

Software  Quality  Factors: 
[Correctness 

Beneficial  Quality  Factors: 
[Maintainability 
Reusability 

Adverse  Quality  Factors: 


GyMebQQk  Ref erences: 


[@GB:  RADC  Software  Quality  Metric  Worksheets  99.5 

@GB:  SEI  Assessment  of  Software  Engineering  Tools  99.6 

Whole  APSE  Assessment  Issues  3.. 

@GB:  APSE  Characterization  13.1 

Whole  APSE  Assessment  Issues  3., 

@GB:  Ada-Europe  Ada  Environment  Questionnaires  13.2 

Text  Editing  7. 1.1.1, 

@GB:  Text  Editing  Capabilities  Checklist  99.1 

Text  Editing  7. 1.1.1, 

@GB:  Language-Sensitive  Editing  Capabilities  Checklist  99.2 


Systems  Requirements  7. 1.6.1, 

@GB:  NADC/SPS  CASE  Tools  Evaluation 
Systems  Requirements  7. 1.6.1, 

@GB:  Time-Critical  Applications  Support  Checklist 
Software  Requirements  7. 1.6. 2, 

@GB:  NADC/SPS  CASE  Tools  Evaluation 
Software  Requirements  7.1. 6. 2, 

@GB:  Time-Critical  Applications  Support  Checklist 

I 


9.4 

9.5 
9.4 
9.5: 


I 


6.2.1] 

6.2.2, 

6.3.3] 
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Preliminary  Design  7.1. 6. 4, 

@GB:  NADC/SPS  CASE  Tools  Evaluation  9.4; 

Preliminary  Design  7. 1.6. 4, 

@GB:  Time-Critical  Applications  Support  Checklist  9.5; 

Detailed  Design  7. 1.6. 5, 

@GB:  Time-Critical  Applications  Support  Checklist  9.5; 

Assembling  7. 1.6. 6, 

@GB:  Cross  Development  System  Support  Questionnaire  13.3; 

Compilation  7. 1.6.7, 

@GB:  ACVC  5.1; 

Compilation  7. 1.6.7, 

@GB:  Cross  Development  System  Support  Questionnaire  13.3; 

Linking/Loading  7.1.6.13, 

@GB:  Cross  Development  System  Support  Questionnaire  13.3; 

Database  Management  7.2. 1.1, 

@GB:  Database  Management  Checklist  15.2; 

File  Management  7.2. 1.3, 

@GB:  File  Management  Checklist  15.1; 

Electronic  Mail  7.2. 1.4, 

@GB:  Electronic  Mail  Checklist  15.3; 

Program  Library  Management  7.2. 1.7, 

@GB:  Program  Library  Management  Checklist  5.9; 

Performance  Monitoring  7.2.1.10, 

@GB:  Performance  Monitoring  Checklist  99.3; 

Configuration  Management  7.2. 2. 7, 

@GB:  Configuration  Management  Capabilities  Checklist  10.1; 

Configuration  Management  7.2. 2. 7, 

@GB:  Configuration  Management  Assessment  Questionnaire  10.3; 

Kernel  7.2.3.3, 

@GB:  CIVC  8.1; 

Runtime  Environment  7.2. 3.5, 

@GB:  ARTEWG  Runtime  Environment  Taxonomy  5.11; 

Runtime  Environment  7. 2. 3. 5, 

@GB:  Runtime  Support  System  Questionnaire  5.14; 

Import/Export  7.2.3. 6, 

@GB:  Import/Export  Capabilities  Checklist  6.3: 

Analysis  7.3, 

@GB:  Testing  Capabilities  Checklist  7.1; 

Syntax  and  Semantics  Checking  7.3.1.15, 

@GB:  Language-Sensitive  Editing  Capabilities  Checklist  99.2; 

Requirements  Prototyping  7.3.2.2, 

@GB:  Requirements  Prototyping  Capabilities  Checklist  9.2; 

Simulation  and  Modeling  7.3. 2. 3, 

@GB:  Instruction-Level  Simulation  Checklist  6.8; 
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Simulation  and  Modeling  7.3. 2. 3. 

@GB:  Simulation  and  Modeling  Capabilities  Checklist  9.3; 

Simulation  and  Modeling  7. 3. 2. 3, 

@GB:  Cross  Development  System  Support  Questionnaire  13.3 
Debugging  7.3. 2. 5, 

@GB:  Debugging  Capabilities  Checklist  6.5; 

Debugging  7.3. 2.5, 

@GB:  Cross  Development  System  Support  Questionnaire  13.3 
Emulation  7.3.2.13, 

@GB:  Cross  Development  System  Support  Questionnaire  13.3 
Timing  Analysis  7.3.2.14, 

@GB:  Timing  Analysis  Capabilities  Checklist  6.6; 

Timing  Analysis  7.3.2.14, 

@GB:  Cross  Development  System  Support  Questionnaire  13.3 
Real-Time  Analysis  7.3.2.17, 


@GB:  Real-Time  Analysis  Capabilities  Checklist  6.7] 
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6.4.10  Consistency 


Description: 

Those  characteristics  of  software  which  provide  for  uniform  design  and  implementation  tech¬ 
niques  and  notation.  Metrics  include  the  use  of  conventions  for:  design  representations,  call¬ 
ing  sequences,  communication  protocols,  error  handling,  object  references,  coding  style, 
data  representations,  naming  conventions,  global  data,  and  multiple  object  copies.  [@RADC 
1985] 


Cross  References: 


Software  Quality  Factors: 

[Correctness 

6.2.1, 

Maintainability 

6.2.2] 

Beneficial  Quality  Factors: 

[Verifiability,  Testability 

6.2.3, 

Expandability.  Flexibility 

6.3.1, 

Reusability 

6.3.3] 

Adverse  Quality  Factors: 


Guidebook  References: 


[@GB:  RADC  Software  Quality  Metric  Worksheets  99.5: 

@GB:  SEI  Assessment  of  Software  Engineering  Tools  99.6: 

Whole  APSE  Assessment  Issues  3., 

@GB:  Distributed  APSE  Questionnaire  12.1; 

Compilation  7. 1.6. 7, 

@GB:  Compiler  Assessment  Questionnaire  5.12; 

Configuration  Management  7. 2.2.7, 


@GB:  Configuration  Management  Assessment  Questionnaire  10.3: 
Command  Language  Processing  7.2.3. 1, 

@GB:  Command  Language  Interpreter 

Assessment  Questionnaire  99.4] 
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6.4.11  Cost 


Description: 

The  cost  of  a  complete  component  or  the  costs  of  features  of  a  component.  The  cost  of  a 
component  may  vary  depending  on  delivery  with  source  code  or  object  code  only  (for  exam¬ 
ple).  Other  cost  considerations  are  installation,  user  assistance,  and  maintenance  support. 
[@E&V  Requirements] 


Cross  References: 

Software  Quality  Factors: 

[Usability  6.1.5] 

Beneficial  Quality  Factors: 

Adverse  Quality  Factors: 

GuiC-^QQk-References: 


[@GB:  Vendor  Evaluation  Questionnaire  99.7; 

@GB:  Cost  Questionnaire  99.9; 

Whole  APSE  Assessment  Issues  3., 

@GB:  APSE  Characterization  13.1: 

Compilation  7. 1.6.7, 

@GB:  Compiler  Assessment  Questionnaire  5.12; 

Configuration  Management  7.2. 2. 7, 


@GB:  Configuration  Management  Assessment  Questionnaire  10.3: 
Command  Language  Processing  7.2. 3.1, 

@GB:  Command  Language  Interpreter 

Assessment  Questionnaire  99.4] 


I 
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6.4.12  Distributedness 


Description: 

Those  characteristics  of  software  which  determine  the  degree  to  which  software  functions 
are  geographically  or  logically  separated  within  the  system.  Metrics  include:  data  at  different 
nodes  or  on  different  storage  devices,  data  accessibility  at  different  nodes,  alternative  data 
sources,  functional  redundancy,  distributed  control,  and  minimum  number  of  nodes. 
[@RADC  1985] 

Cross  References: 


Software  Quality  Factors: 


[Survivability 

6.1.4] 

Beneficial  Quality  Factors: 

[Expandability,  Flexibility 

6.3.1] 

Adverse  Quality  Factors: 

[Integrity 

6.1.2] 

Guidebook  References: 

[@GB:  RADC  Software  Quality  Metric  Worksheets 
Whole  APSE  Assessment  Issues 

3., 

99.5: 

@GB:  APSE  Customization  Questionnaire 

13.4] 
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6.4.13  Document  Accessibility 


Description: 

Those  characteristics  of  software  which  provide  for  easy  access  to  software  and  selective 
use  of  its  components.  Metrics  include:  ready  access  to  documents,  clarity  and  simplicity  of 
documents,  depiction  of  control  and  data  flow,  finding  aids,  separation  by  function,  complete 
description  of  requirements  and  limitations,  and  access  to  the  source  code  or  runtime  ver¬ 
sion.  [@RADC  1985] 

Cross  References: 


Software  Quality  Factors: 

[Reusability  6.3.3] 


Beneficial  Quality  Factors: 

[Usability  6.1.5, 

Maintainability  6.2.2] 


Adverse  Quality  Factors: 

[Integrity  6.1.2] 


Guidebook  References: 


[@GB:  RADC  Software  Quality  Metric  Worksheets  99.5: 

@GB:  Vendor  Evaluation  Questionnaire  99.7; 

Compilation  7. 1.6. 7, 

@GB:  Compiler  Assessment  Questionnaire  5.12; 

Configuration  Management  7.2. 2. 7, 

@GB:  Configuration  Management  Assessment  Questionnaire  10.3; 
Command  Language  Processing  7.2.3. 1, 

@GB:  Command  Language  Interpreter 

Assessment  Questionnaire  99.4] 
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6.4.14  Functional  Overlap 


Description: 

Those  characteristics  of  software  which  provide  common  functions  to  both  systems. 
[@RADC  1985] 


Cross  References: 

Software  Quality  Factors: 

[Interoperability  6.3.2] 

Beneficial  Quality  Factors: 

Adverse  Quality  Factors: 


Guidebook  References: 

[@GB:  RADC  Software  Quality  Metric  Worksheets  99.5: 

@GB:  SEI  Assessment  of  Software  Engineering  Tools  99.6; 

Whole  APSE  Assessment  Issues  3., 

@GB:  Distributed  APSE  Questionnaire  12.1] 
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6.4.15  Functional  Scope 


Description: 

Those  characteristics  of  software  which  provide  commonality  of  functions  among  applica 
tions.  Metrics  include:  function  specificity,  function  commonality,  and  function  selective  us 
ability.  [@RADC  1985) 


Cross  References: 


Software  Quality  Factors: 
[Reusability 


6.3.3] 


Beneficial  Quality  Factors: 


Adverse  Quality  Factors: 


Guidebook  References: 

[@GB:  RADC  Software  Quality  Metric  Worksheets  99.5: 

Runtime  Environment  7.2.3. 5, 

@GB:  Runtime  Support  System  Questionnaire  5.14] 
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6.4.16  Generality 


Description: 

Those  characteristics  of  software  which  provide  breadth  to  the  functions  performed  with  re¬ 
spect  to  the  application.  Metrics  include:  the  number  of  calling  modules,  modules  that  per¬ 
form  only  a  single  process  (external  input,  external  output,  algorithm),  modules  free  of 
machine-dependent  operations,  limitations  on  data  volume,  and  limitations  on  data  values. 
[@RADC  1985] 


Cross  References: 

Software  Quality  Factors: 


[Expandability,  Flexibility 

6.3.1, 

Reusability 

6.3.3] 

Beneficial  Quality  Factors: 

[Interoperability 

6.3.2] 

Adverse  Quality  Factors: 

[Efficiency 

6.1.1, 

Integrity 

6.1.2, 

Reliability 

6.1.3, 

Survivability 

6.1.4] 

Guidebook  References: 


[@GB:  RADC  Software  Quality  Metric  Worksheets 
Whole  APSE  Assessment  Issues 

3., 

99.5: 

@GB:  APSE  Customization  Questionnaire 

13.4; 

Compilation 

7.1. 6.7, 

@GB:  Compiler  Assessment  Questionnaire 

5.12: 

Configuration  Management 

7. 2. 2. 7, 

@GB:  Configuration  Management  Assessment  Questionnaire 

10.3; 

Command  Language  Processing 

7. 2. 3.1, 

@GB:  Command  Language  Interpreter 

Assessment  Questionnaire 

99.4; 

Runtime  Environment 

7.2. 3. 5, 

@GB:  Runtime  Support  System  Questionnaire 

5.14] 
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6.4.17  Granularity 


Description: 

The  degree  to  which  a  component  has  separate  limited  functions  that  are  composable,  user 
selectable,  and  communicate  through  a  common  database.  [@E&V  Requirements] 


Cross  References: 


Software  Quality  Factors: 

[Survivability 

6.1.4, 

Usability 

6.1.5, 

Maintainability 

6.2.2. 

Verifiability,  Testability 

6.2.3, 

Expandability,  Flexibility 

6.3.1, 

Reusability 

6.3.3] 

Beneficial  Quality  Factors: 
Adverse  Quality  Factors: 


Guidebook  References: 

[@GB:  SEI  Assessment  of  Software  Engineering  Tools  99.6: 

Compilation  7. 1.6. 7, 

@GB:  Compiler  Assessment  Questionnaire  5.12; 

Configuration  Management  7. 2. 2. 7, 

@GB:  Configuration  Management  Assessment  Questionnaire  10.3: 
Command  Language  Processing  7.2.3. 1, 

@GB:  Command  Language  Interpreter 

Assessment  Questionnaire  99.4; 

Runtime  Environment  7. 2. 3. 5, 

@GB:  Runtime  Support  Syste  Questionnaire  5.14] 
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6.4.18  Maturity 


Description: 

The  extent  to  which  a  component  has  been  used  in  the  development  of  deliverable  software 
by  typical  users  and  to  which  the  feedback  from  that  use  has  been  reflected  in  modifications 
to  the  component.  [@E&V  Requirements] 


Cross  References: 


Software  Quality  Factors: 

[Reliability  6.1.3, 

Usability  6.1.5] 


Beneficial  Quality  Factors: 


Adverse  Quality  Factors: 


Guidebook  References: 


[@GB:  SEI  Assessment  of  Software  Engineering  Tools  99.6: 

@GB:  Vendor  Evaluation  Questionnaire  99.7; 

@GB:  Maturity  Questionnaire  99.10; 

Whole  APSE  Assessment  Issues  3., 

@GB:  APSE  Characterization  13.1: 

Compilation  7. 1.6. 7, 

@GB:  Compiler  Assessment  Questionnaire  5.12; 

Configuration  Management  7. 2. 2. 7, 


@GB:  Configuration  Management  Assessment  Questionnaire  10.3: 
Command  Language  Processing  7.2.3. 1, 

@GB:  Command  Language  Interpreter 

Assessment  Questionnaire  99.4] 
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6.4.19  Modularity 


Description: 

The  extent  to  which  software  is  composed  of  discrete  components  such  that  a  change  to  one 
component  has  minimal  impact  on  other  components.  [@IEEE  1983]  The  extent  to  which  a 
component  is  implemented  in  a  hierarchical  structure  in  which  identifiable  functions  are  iso¬ 
lated  in  separate  compilation  units.  Modular  software  is  characterized  by  low  coupling  be¬ 
tween  units  (restrictions  on  the  number  and  types  of  parameters)  and  high  cohesion  within 
units  (singularity  of  purpose). 


Cross  References: 


Software  Quality  Factors: 


[Survivability 

6.1.4, 

Maintainability 

6.2.2, 

Verifiability,  Testability 

6.2.3, 

Expandability,  Flexibility 

6.3.1, 

Interoperability 

6.3.2, 

Reusability 

6.3.3, 

T  ransportability 

6.3.4] 

Beneficial  Quality  Factors: 

Adverse  Quality  Factors: 

[Efficiency 

6.1.1] 

Guidebook  References: 


[<®GB:  RADC  Software  Quality  Metric  Worksheets 
Whole  APSE  Assessment  Issues 

3., 

99.5; 

<®GB:  APSE  Customization  Questionnaire 

13.4; 

Runtime  Environment 

7. 2. 3. 5, 

@GB:  Runtime  Support  System  Questionnaire 

5.14] 
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6.4.20  Operability,  Communicativeness 


Description: 

Those  characteristics  of  software  which  determine  operations  and  procedures  concerned 
with  operation  of  software  and  which  provide  useful  inputs  and  outputs  which  can  be  assimi¬ 
lated.  Metrics  include:  the  user/component  dialog,  error  reporting  and  response  procedures, 
debugging  facilities,  ergonomic  considerations,  event  logging  or  monitoring  facilities,  trans¬ 
parency  of  implementation  details,  and  input  and  output  options.  [@RADC  1985]  The  user/ 
component  dialog  established  to  control  the  execution  of  the  component.  This  is  driven  by 
the  set  of  assumptions  made  by  the  component  and  made  about  the  component  by  the  per¬ 
sons  who  use  it. 


Cross  References: 


Software  Quality  Factors: 

[Usability  6.1.5] 

Beneficial  Quality  Factors: 

[Maintainability  6.2.2, 

Verifiability,  Testability  6.2.3] 

Adverse  Quality  Factors: 

[Efficiency  6.1.1] 
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Guidebook  References: 


[@GB:  RADC  Software  Quality  Metric  Worksheets  99.5 

@GB:  SEI  Assessment  of  Software  Engineering  Tools  99.6 

Whole  APSE  Assessment  Issues  3., 

@GB:  Distributed  APSE  Questionnaire  12.1 

Whole  APSE  Assessment  Issues  3.. 

@GB:  APSE  Characterization  13.1; 

Whole  APSE  Assessment  Issues  3., 

@GB:  Ada-Europe  Ada  Environment  Questionnaires  13.2 

Whole  APSE  Assessment  Issues  3., 

@GB:  APSE  Customization  Questionnaire  13.4 

Compilation  7. 1.6. 7, 

@GB:  Compiler  Assessment  Questionnaire  5.12; 

Configuration  Management  7.2. 2.7, 


@GB:  Configuration  Management  Assessment  Questionnaire  10.3; 
Command  Language  Processing  7.2. 3.1, 

@GB:  Command  Language  Interpreter 

Assessment  Questionnaire  99.4] 
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6.4.21  Power 


Description: 

The  extent  to  which  a  component  has  capabilities,  such  as  default  options  and  wild  card  op¬ 
erations,  that  contribute  to  the  effectiveness  of  the  user.  [@E&V  Requirements] 


Cross  References: 


Software  Quality  Factors: 

[Usability  6.1.5] 


Beneficial  Quality  Factors: 


Adverse  Quality  Factors: 


Guidebook  References; 


[@GB:  SEI  Assessment  of  Software  Engineering  Tools  99.6: 

Whole  APSE  Assessment  Issues  3., 

@GB:  APSE  Characterization  13.1: 

Whole  APSE  Assessment  Issues  3., 

@GB:  Ada-Europe  Ada  Environment  Questionnaires  13.2; 

Text  Editing  7. 1.1.1, 

@GB:  Text  Editing  Capabilities  Checklist  99.1; 

Text  Editing  7. 1.1.1, 

@GB:  Language-Sensitive  Editing  Capabilities  Checklist  99.2: 

Preliminary  Design  7. 1.6. 4, 

@GB:  SEI  Design  Support  Equipment  9.1; 

Detailed  Design  7. 1.6. 5, 

@GB:  SEI  Design  Support  Equipment  9.1; 

Assembling  7. 1.6. 6, 

@GB:  Assembling  Checklist  6.1; 

Compilation  7. 1.6. 7, 

@GB:  Compilation  Checklist  5.8; 
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Compilation 

7.1. 6.7, 

@GB:  Compiler  Assessment  Questionnaire 

5.12; 

Linking/Loading 

7.1.6.13, 

@GB:  Linking/Loading  Checklist 

6.2; 

Database  Management 

7.2.1. 1. 

@GB:  Database  Management  Checklist 

15.2 

File  Management 

7. 2. 1.3, 

@GB:  File  Management  Checklist 

15.1 

Electronic  Mail 

7.2.1. 4, 

@GB:  Electronic  Mail  Checklist 

15.3 

Program  Library  Management 

7.2.1. 7, 

@GB:  Program  Library  Management  Checklist 

5.9; 

Configuration  Management 

7.2.2 .7, 

@GB:  SEI  Configuration  Management  Experiment 

10.2; 

Configuration  Management 

7. 2. 2. 7, 

@GB:  Configuration  Management  Assessment  Questionnaire 

10.3; 

Command  Language  Processing 

7.2.3. 1 , 

@GB:  Command  Language  Interpreter 

Assessment  Questionnaire 

99.4; 

Analysis 

7.3, 

@GB:  Testing  Capabilities  Checklist 

7.1; 

Syntax  and  Semantics  Checking 

7.3.1.15, 

@GB:  Language-Sensitive  Editing  Capabilities  Checklist 

99.2; 

Dynamic  Analysis 

7.3.2, 

@GB:  SEI  Unit  Testing  and  Debugging  Experiment 

7.2; 

Debugging 

7. 3. 2. 5, 

@GB:  Debugging  Capabilities  Checklist 

6.5; 

Debugging 

7. 3. 2. 5, 

@GB:  SEI  Unit  Testing  and  Debugging  Experiment 

7.2; 

Emulation 

7.3.2.13, 

@GB:  Emulation  Capabilities  Checklist 

6.4] 
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6.4.22  Processing  (Execution)  Effectiveness 


Description: 

Those  characteristics  of  the  software  which  provide  for  minimum  utilization  of  processing  re¬ 
sources  in  performing  functions.  Metrics  include  algorithmic  processing  effectiveness  and 
data  usage  effectiveness.  [@RADC  1985]  The  choice  between  alternative  algorithms  based 
on  those  taking  the  least  amount  of  time.  [@DACS  1979] 


Cross  References: 

Software  Quality  Factors: 


[Efficiency 

6.1.1] 

Beneficial  Quality  Factors: 

Adverse  Quality  Factors: 

[Maintainability 

6.2.2, 

Verifiability,  Testability 

6.2.3, 

Transportability 

6.3.4] 

Guidebook  References: 

[@GB:  RADC  Software  Quality  Metric  Worksheets 

99.5: 

@GB:  SEI  Assessment  of  Software  Engineering  Tools 

99.6: 

Whole  APSE  Assessment  Issues 

3., 

@GB:  Ada-Europe  Ada  Environment  Questionnaires 

13.2: 

Systems  Requirements 

7. 1.6.1, 

@GB:  NADC/SPS  CASE  Tools  Evaluation 

9.4; 

Software  Requirements 

7.1 .6.2, 

@GB:  NADC/SPS  CASE  Tools  Evaluation 

9.4; 

Preliminary  Design 

7.1 .6.4, 

@GB:  SEI  Design  Support  Experiment 

9.1; 

Preliminary  Design 

7.1. 6.4, 

@GB:  NADC/SPS  CASE  Tools  Evaluation 

9.4; 

Detailed  Design 

7.1. 6.5, 

@GB:  SEI  Design  Support  Experiment 

9.1: 
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’Compilation  7. 1.6. 7. 

@GB:  IDA  Benchmarks  5.2; 

’Compilation  7.1. 6. 7, 

@GB:  ACEC  5.3; 

’Compilation  7. 1.6. 7, 

@GB:  PIWG  Benchmark  Tests  5.4; 

’Compilation  7. 1.6. 7, 

@GB:  University  of  Michigan  Benchmark  Tests  5.5; 

’Compilation  7. 1.6. 7, 

@GB:  ARTEWG  Catalogue  of  Ada  Runtime 

Implementation  Dependencies  5.10; 

Compilation  7. 1.6. 7, 

@GB:  Compiler  Assessment  Questionnaire  5.12; 

Configuration  Management  7.2. 2.7, 

@GB:  SEI  Configuration  Management  Experiment  10.2; 

Configuration  Management  7. 2.2.7, 

@GB:  Configuration  Management  Assessment  Questionnaire  10.3; 

Command  Language  Processing  7.2. 3.1, 

@GB:  Command  Language  Interpreter 

Assessment  Questionnaire  99.4; 

Dynamic  Analysis  7.3.2, 

@GB:  SEI  Unit  Testing  and  Debugging  Experiment  7.2; 

Debugging  7.3.2.5, 

@GB:  SEI  Unit  Testing  and  Debugging  Experiment  7.2] 


’NOTE;  Normally,  the  concern  of  evaluation  is  the  quality  of  a  tool.  However,  this 
evaluation  technique  focuses  on  the  products  of  the  tool  rather  than  the  tool  itself. 
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6.4.23  Proprietary  Rights 


Description: 

Restrictions  on  the  release,  distribution,  or  use  of  a  component.  This  includes  so  called 
“data  rights"  restrictions.  [@E&V  Requirements] 


Cross  References: 


Software  Quality  Factors: 

[Maintainability  6.2.2, 

Expandability.  Flexibility  6.3.1] 


Beneficial  Quality  Factors: 


Adverse  Quality  Factors: 


Guidebook  ^References: 


[@GB:  SEI  Assessment  of  Software  Engineering  Tools  99.6: 

@GB:  Vendor  Evaluation  Questionnaire  99.7; 

@GB:  Licensing  Issues  Questionnaire  99.11; 

Whole  APSE  Assessment  Issues  3., 

@GB:  Ada-Europe  Ada  Environment  Questionnaires  13.2; 

Compilation  7. 1.6.7, 

@GB:  Compiler  Assessment  Questionnaire  5.12; 

Configuration  Management  7.2. 2. 7, 


@GB:  Configuration  Management  Assessment  Questionnaire  10.3; 
Command  Language  Processing  7.2.3. 1, 

@GB:  Command  Language  Interpreter 

Assessment  Questionnaire  99.4] 
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6.4.24  Reconfigurability 


Description: 

Those  characteristics  of  software  which  provide  for  continuity  of  system  operation  when  one 
or  more  processors,  storage  units,  or  communication  links  fail.  Metrics  include  ensuring 
communication  continuity,  maintaining  the  integrity  of  data  values,  restoring  failed  communi¬ 
cation  links,  and  replicating  critical  data  at  multiple  nodes.  [@RADC  1985] 


Cross  References: 


Software  Quality  Factors: 

[Survivability  6.1.4] 

Beneficial  Quality  Factors: 

[Maintainability  6.2.2] 

Adverse  Quality  Factors: 

[Efficiency  6.1.1, 

Expandability,  Flexibility  6.3.1, 

Reusability  6.3.3, 

Transportability  6.3.4] 


Guidebook  References: 

[<®GB:  RADC  Software  Quality  Metric  Worksheets  99.5] 
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6.4.25  Rehostability 


Description: 

Those  characteristics  of  software  which  determine  its  non-dependency  on  software  environ¬ 
ment  (computing  system,  operating  system,  utilities,  input,  output  routines,  libraries). 
[@RADC  1985:  Independence]  The  ability  of  an  APSE  component  to  be  installed  on  a  differ¬ 
ent  host  or  different  operating  system  with  needed  reprogramming  localized  to  the  KAPSE  or 
machine  dependencies.  Included  in  this  attribute  is  conformance  to  any  existing  pertinent  in¬ 
terface  standards  such  as  the  CAIS.  [@E&V  Requirements] 


Cross  References: 


Software  Quality  Factors: 


[Interoperability 

6.3.2, 

Reusability 

6.3.3, 

Transportability 

6.3.4] 

Beneficial  Quality  Factors: 

Adverse  Quality  Factors: 

[Efficiency 

6.1.1] 

Guidebook  References: 


[@GB:  Host  and  Target  Questionnaire  14.1: 

@GB:  Machine-Specific  Characteristics  Questionnaire  14.2; 

@GB:  RADC  Software  Quality  Metric  Worksheets  99.5: 

@GB:  SEI  Assessment  of  Software  Engineering  Tools  99.6; 

Compilation  7. 1.6. 7, 

@GB:  Compiler  Assessment  Questionnaire  5.12; 

Configuration  Management  7.2. 2. 7, 


@GB:  Configuration  Management  Assessment  Questionnaire  10.3: 
Command  Language  Processing  7.2. 3.1, 

@GB:  Command  Language  Interpreter 

Assessment  Questionnaire  99.4] 
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6.4.26  Required  Configuration 


i 

Description:  I 

The  amount  and  types  of  hardware  or  software  facilities  needed  for  the  operation  of  a  i 

component.  This  includes  primary  and  secondary  storage  and  any  other  required  re-  I 

sources.  [@E&V  Requirements] 


Cross  References: 


Software  Quality  Factors: 

[Usability  6.1.5] 


Beneficial  Quality  Factors: 

[Expandability,  Flexibility  6.3.1, 

Reusability  6.3.3, 

Transportability  6.3.4] 


Adverse  Quality  Factors: 


Guidebook  References: 


[@GB:  Required  Configuration  Questionnaire 

Whole  APSE  Assessment  Tools 

3., 

99.8; 

@GB:  Distributed  APSE  Questionnaire 

12.1; 

Whole  APSE  Assessment  Issues 

3., 

@GB:  APSE  Characterization 

13.1; 

Whole  APSE  Assessment  Issues 

3., 

@GB:  Ada-Europe  Ada  Environment  Questionnaires 

13.2; 

Whole  APSE  Assessment  Issues 

3., 

@GB:  APSE  Customization  Questionnaire 

13.4; 

Compilation 

7.1 .6.7, 

@GB:  Compiler  Assessment  Questionnaire 

5.12; 

Command  Language  Processing 

7, 2.3.1, 

@GB:  Command  Language  Interpreter 

Assessment  Questionnaire 

99.4] 
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6.4.27  Retargetability 
Description: 

The  ability  of  an  APSE  component  to  accomplish  its  function  with  respect  to  another  target. 
The  component  may  require  modification.  [@E&V  Requirements]  The  retargetability  of  a 
component  is  usually  determined  by  the  software  and  machine  independence  of  the  outputs 
generated  by  the  component. 


Cross  References: 


Software  Quality  Factors: 


[Expandability,  Flexibility 

6.3.1, 

Interoperability 

6.3.2, 

Reusability 

6.3.3, 

Transportability 

6.3.4] 

Beneficial  Quality  Factors: 

Adverse  Quality  Factors: 

[Efficiency 

6.1.1] 

Guidebook  Referensgs: 

[@GB:  Host  and  Target  Questionnaire 
@GB:  Machine-Specific  Characteristics  Questionnaire 
@GB:  RADC  Software  Quality  Metric  Worksheets 
Whole  APSE  Assessment  Issues  3.. 

@GB:  APSE  Customization  Questionnaire 
'Compilation  7. 1.6. 7, 

@GB:  ARTEWG  Catalogue  of  Ada 

Runtime  Implementation  Dependencies 
Compilation  7. 1.6. 7. 

@GB:  Compiler  Assessment  Questionnaire 
Runtime  Environment  7.2. 3. 5, 

@GB:  Runtime  Support  System  Questionnaire 


14.1; 

14.2: 

99.5: 

13.4; 


5.10: 

5.12: 

5.14] 


'NOTE:  Normally,  the  concern  of  evaluation  is  the  quality  of  a  tool.  However,  this 
evaluation  technique  focuses  on  the  products  of  the  tool  rather  than  the  tool  itself. 
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6.4.28  Self-Descriptiveness 


Description: 

Those  characteristics  of  software  which  provide  explanation  of  the  implementation  of  func¬ 
tions.  Metrics  include:  quantity  and  effectiveness  of  comments  and  the  descriptiveness  of 
the  implementation  language  (such  as  meaningful  variable  names  and  formatting  that  re¬ 
flects  the  structure  of  code).  [@RADC  1985]  The  technical  data,  including  on-line,  docu¬ 
mentation,  listings,  and  printouts,  which  serve  the  purpose  of  elaborating  the  design  or  details 
of  a  component.  [@DACS  1979] 


Cross  References: 


Software  Quality  Factors: 


[Maintainability 

6.2.2, 

Verifiability,  Testability 

6.2.3, 

Expandability,  Flexibility 

6.3.1, 

Reusability 

6.3.3, 

Transportability 

6.3.4] 

Beneficial  Quality  Factors: 

Adverse  Quality  Factors: 

[Efficiency 

6.1.1] 

99.5: 


Guidebook  References: 

[r®GB:  RADC  Software  Quality  Metric  Worksheets 
Systems  Requirements  7. 1.6.1, 

@GB:  NADC/SPS  CASE  Tools  Evaluation 
Software  Requirements  7. 1.6. 2, 

@GB:  NADC/SPS  CASE  Tools  Evaluation 
Preliminary  Design  7. 1.6. 4, 

@GB:  NADC/SPS  CASE  Tools  Evaluation 
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6.4.29  Simplicity 


Description: 

Those  characteristics  of  software  which  provide  for  definition  and  implementation  of  func¬ 
tions  in  the  most  non-complex  and  understandable  manner.  Metrics  include:  design  struc¬ 
ture,  use  of  a  structured  language,  data  and  control  flow  complexity,  coding  simplicity, 
specificity,  and  Halstead’s  level  of  difficulty  measure.  [@RADC  1985]  The  lack  of  complica¬ 
tion  of  a  system  or  system  component,  determined  by  such  factors  as  the  number  and  intri¬ 
cacy  of  interfaces,  the  number  and  intricacy  of  conditional  branches,  the  degree  of  nesting, 
the  types  of  data  structures,  and  other  system  characteristics.  [@IEEE  1983] 


Cross  References: 


Software  Quality  Factors: 


[Reliability 

6.1.3, 

Maintainability 

6.2.2, 

Verifiability,  Testability 

6.2.3, 

Expandability.  Flexibility 

6.3.1, 

Reusability 

6.3.3] 

Beneficial  Quality  Factors: 

Adverse  Quality  Factors: 

[Efficiency 

6.1.1] 

Guidebook  References: 

[@GB:  RADC  Software  Quality  Metric  Worksheets  99.5: 

@GB:  SEI  Assessment  of  Software  Engineering  Tools  99.6] 
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6.4.30  Software  Production  Vehicle  (s) 


Description: 

The  methodology (ies),  language(s),  technique (s),  and  software  tool(s)  used  to  produce  the 
software  related  to  a  component.  [@E&V  Requirements] 


Cross  References: 


Software  Quality  Factors: 

[Maintainability  6.2.2, 

Expandability,  Flexibility  6.3.1, 

Reusability  6.3.3, 

Transportability  6.3.4] 


Beneficial  Quality  Factors: 
Adverse  Quality  Factors: 


Guidebook  References: 

[@GB:  SEI  Assessment  of  Software  Engineering  Tools  99.6; 

@GB:  Vendor  Evaluation  Questionnaire  99.7; 

@GB:  Software  Production  Vehicle(s)  Questionnaire  99.12] 
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6.4.31  Storage  Effectiveness 


Description: 

Those  characteristics  of  the  software  which  provide  for  minimum  utilization  of  storage  re¬ 
sources  in  performing  functions.  Metrics  include:  the  use  of  virtual  storage,  dynamic  reallo¬ 
cation  of  memory,  and  the  avoidance  of  redundant  storage  of  data.  [@RADC  1985]  The 
choice  between  alternative  source  code  constructions  based  on  those  taking  the  minimum 
number  of  words  of  object  code  or  in  which  the  information-packing  ...  is  high.  [@DACS 
1979] 

Cross  References: 

Software  Quality  Factors: 

[Efficiency 

Beneficial  Quality  Factors: 

Adverse  Quality  Factors: 

[Maintainability 
Verifiability,  Testability 
Transportability 


Guidebook  References: 


[@GB:  RADC  Software  Quality  Metric  Worksheets 

99.5: 

@GB:  SEI  Assessment  of  Software  Engineering  Tools 

99.6; 

Preliminary  Design 

7.1 .6.4, 

@GB:  SEI  Design  Support  Experiment 

9.1; 

Detailed  Design 

7.1 .6.5, 

(6GB:  SEI  Design  Support  Experiment 

9.1; 

'Compilation 

7. 1.6. 7, 

(6GB:  IDA  Benchmarks 

5.2; 

'Compilation 

7.1. 6.7, 

(6GB:  AC  EC 

5.3; 

"Compilation 

7.1. 6. 7, 

(6GB:  PIWG  Benchmark  Tests 

5.4; 

'Compilation 

7.1 .6.7, 

<6GB:  ARTEWG  Catalogue  of  Ada 

Runtime  Implementation  Dependencies 

5.10; 

'NOTE:  Normally,  the  concern  of  evaluation  is  the  quality  of  a  tool. 

However,  this 

evaluation  technique  focuses  on  the  products  of  the  tool  rather  than  the  tool  itself. 


6.1.1] 


6.2.2, 

6.2.3, 

6.3.4] 
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Compilation  7. 1.6. 7, 

@GB:  Compiler  Assessment  Questionnaire  5.12; 

Configuration  Management  7. 2.2.7, 

@GB:  Configuration  Management  Assessment  Questionnaire  10.3; 
Command  Language  Processing  7.2. 3.1, 

@GB:  Command  Language  Interpreter 

Assessment  Questionnaire  99.4] 
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6.4.32  System  Accessibility 


Description: 

Those  characteristics  of  software  which  provide  for  control  and  audit  of  access  to  the  soft¬ 
ware  and  data.  [@RADC  1985)  Hardware  or  software  features,  operating  procedures,  or 
management  procedures  designed  to  permit  authorized  access  and  prevent  unauthorized 
access  to  a  computer  system.  [@IEEE  1983] 


Cross  References; 


Software  Quality  Factors: 

[Integrity  6.1.2] 


Beneficial  Quality  Factors: 


Adverse  Quality  Factors: 

[Efficiency  6.1.1] 

Guidebook  References: 

99.5: 
99.6: 

13.2; 

5.14] 


[@GB:  RADC  Software  Quality  Metric  Worksheets 
@GB:  SEI  Assessment  of  Software  Engineering  Tools 
Whole  APSE  Assessment  Issues  3., 

@GB:  Ada-Europe  Ada  Environment  Questionnaires 
Runtime  Environment  7.2. 3.5, 

@GB:  Runtime  Support  System  Questionnaire 
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6.4.33  System  Clarity 


Description: 

Those  characteristics  of  software  which  provide  for  clear  description  of  program  structure  in 
a  non-complex  and  understandable  manner.  Metrics  include:  interface  complexity,  program 
flow  complexity,  application  functional  complexity,  communication  complexity,  and  structure 
clarity.  [@RADC  1985] 

Cross  References: 

Software  Quality  Factors: 

[Reusability  6.3.3] 

Beneficial  Quality  Factors: 

Adverse  Quality  Factors: 


Guidebook  References: 


[@GB:  RADC  Software  Quality  Metric  Worksheets 
Systems  Requirements 

7.1. 6.1, 

99.5; 

(a)GE:  NADC/SPS  CASE  Tools  Evaluation 

9.4; 

Software  Requirements 

7.1. 6.2, 

@GB:  NADC/SPS  CASE  Tools  Eva.  ation 

94; 

Preliminary  Design 

7.1 .6.4, 

@GB:  NADC/SPS  CASE  Tools  Evaluation 

9.4] 
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6.4.34  System  Compatibility 

Description; 

Those  characteristics  of  software  which  provide  the  hardware,  software,  and  communication 
compatibility  of  two  systems.  Metrics  include:  communication  compatibility,  data  compatibili¬ 
ty,  hardware  compatibility,  software  compatibility,  and  access  to  documentation  for  the  other 
systems.  [@RADC  1985] 


Cross  References; 

Software  Quality  Factors: 


[Interoperability 

6.3.2] 

Beneficial  Quality  Factors: 

Adverse  Quality  Factors: 

[Integrity 

6.1.2] 

Guidebook  References; 

[@GB:  RADC  Software  Quality  Metric  Worksheets 

99.5: 

@GB:  SEI  Assessment  of  Software  Engineering  Tools 

99.6; 

Whole  APSE  Assessment  Tools 

3., 

@GB:  Distributed  APSE  Questionnaire 

12.1: 

Whole  APSE  Assessment  Issues 

3., 

@GB:  APSE  Customization  Questionnaire 

13.4: 

Compilation 

7.1. 6.7, 

@GB:  Compiler  Assessment  Questionnaire 

5.12; 

Configuration  Management 

7. 2. 2. 7, 

@GB:  Configuration  Management  Assessment  Questionnaire 

10.3; 

Command  Language  Processing 

7.2. 3.1, 

@GB:  Command  Language  Interpreter 

Assessment  Questionnaire 

99.4; 

Runtime  Environment 

7.2. 3. 5, 

@GB:  Runtime  Support  System  Questionnaire 


5.14] 


J 
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6.4.35  Traceability 


Description: 

Those  characteristics  of  software  which  provide  a  thread  of  origin  from  the  implementa¬ 
tion  to  the  requirements  with  respect  to  the  specified  development  envelope  and  opera¬ 
tional  environment.  [@RADC  1985] 


Cross  References: 

Software  Quality  Factors: 


[Correctness 

6.2.1] 

Beneficial  Quality  Factors: 

[Maintainability 

6.2.2, 

Verifiability.  Testability 

6.2.3. 

Expandability.  Flexibility 

6.3.1. 

Reusability 

6.3.3] 

Adverse  Quality  Factors: 

Guidebook  References: 

[@GB:  RADC  Software  Quality  Metric  Worksheets  99.5: 

@GB:  SEI  Assessment  of  Software  Engineering  Tools  99.6] 
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6.4.36  Training 


Description: 

Those  characteristics  of  software  which  provide  transition  from  current  operation  and  provide 
initial  familiarization.  [@RADC  1985]  The  extent  to  which  training  and  other  user  help  is  avail¬ 
able  from  the  vendor  of  a  component  or  from  the  component  itself,  including  on-line,  docu¬ 
mentation,  listings,  and  printouts,  which  serve  the  purpose  of  providing  operating  instructions 
for  using  the  component  to  obtain  desired  results.  [@DACS  1979] 


Cross  References: 

Software  Quality  Factors: 

[Usability  6.1.5] 

Beneficial  Quality  Factors: 

Adverse  Quality  Factors: 


Guidebook  References: 


[@GB:  RADC  Software  Quality  Metric  Worksheets  99.5: 

@GB:  SEI  Assessment  of  Software  Engineering  Tools  99.6: 

@GB:  Vendor  Evaluation  Questionnaire  99.7; 

Whole  APSE  Assessment  issues  3., 

@GB:  Ada-Europe  Ada  Environment  Questionnaires  13.2; 

Systems  Requirements  7. 1.6.1, 

@GB:  NADC/SPS  CASE  Tools  Evaluation  9.4; 

Software  Requirements  7. 1.6.2, 

@GB:  NADC/SPS  CASE  Tools  Evaluation  9.4; 

Preliminary  Design  7. 1.6. 4, 

@GB:  NADC/SPS  CASE  Tools  Evaluation  9.4; 

Compilation  7. 1.6. 7, 

@GB:  Compiler  Assessment  Questionnaire  5.12; 

Configuration  Management  7.2.2. 7, 


@GB:  Configuration  Management  Assessment  Questionnaire  10.3; 
Command  Language  Processing  7.2. 3.1, 

@GB:  Command  Language  Interpreter 

Assessment  Questionnaire  99.4] 
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6.4.37  Virtuality 


Description: 

Those  characteristics  of  software  which  present  a  system  that  does  not  require  user  knowl¬ 
edge  of  the  physical,  logical,  or  topological  characteristics.  [@RADC  1985] 

Cross  References: 

Software  Quality  Factors: 

[Expandability,  Flexibility  6.3.1] 

Beneficial  Quality  Factors: 

Adverse  Quality  Factors: 

[Efficiency  6.1.1] 

Guidebook  References: 

[@GB:  RADC  Software  Quality  Metric  Worksheets  99.5] 


t 


6-66 


E&V  Reference  Manual,  Version  2.0 


6.4.38  Visibility,  Test  Availability 


Description: 

Those  characteristics  of  software  which  provide  status  monitoring  of  the  development  and 
operation.  [@RADC  1985]  The  availability  of  tests  that  verify  the  correctness  or  effective¬ 
ness  of  a  component  function  or  feature.  These  tests  may  also  verify  proper  response  for  an 
incorrect  input  or  technique.  Metrics  include:  unit  testing,  integration  testing,  and  CSCI  test¬ 
ing.  [@E&V  Requirements] 


Cross  References: 


Software  Quality  Factors: 

[Maintainability  6.2.2. 

Verifiability,  Testability  6.2.3] 

Beneficial  Quality  Factors: 

Adverse  Quality  Factors: 

GuictefeQQK  References: 

99.5: 
99.6; 

5.12; 

10.3: 

99.4] 


[@GB:  RADC  Software  Quality  Metric  Worksheets 
@GB:  SEI  Assessment  of  Software  Engineering  Tools 
Compilation  7. 1.6. 7, 

@GB:  Compiler  Assessment  Questionnaire 
Configuration  Management  7.2.2 .7, 

@GB:  Configuration  Management  Assessment  Questionnaire 
Command  Language  Processing  7.2.3. 1, 

@GB:  Command  Language  Interpreter 
Assessment  Questionnaire 
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FUNCTIONS 


This  chapter  provide?  a  functional  taxonomy  used  for  characterizing  the  functional 
capabilities  of  a  tool,  toolset,  or  APSE.  The  top  two  levels  of  the  taxonomy  are  as  follows: 

•  T  ransformation 

Editing 

Formatting 

On-Line  Assistance  Processing 

Sort/Merge 

Graphics  Generation 

Translation 

Synthesis 

•  Management 

Information  Management 
Project  Management 
Computer  System  Management 

•  Analysis 

Static  Analysis 
Dynamic  Analysis 
Formal  Verification 
Problem  Report  Analysis 
Change  Request  Analysis. 

The  complete,  three-level,  taxonomy  may  be  seen  by  examining  the  Table  of  Contents  under 
Chapter  7. 


This  type  of  functional  classification  scheme  was  first  described  in  the  NBS 
Taxonomy  report  [@Houghton  1983]  and  later  elaborated  in  the  SEE  Taxonomy  [@Kean  1985]. 
The  E&V  Team  has  followed  the  general  scheme  suggested  in  the  above-referenced 
documents,  but  with  some  further  elaboration  and  modifications  designed  to  make  the  taxonomy 
compatible  with  the  rest  of  the  multiple  indexing  scheme  used  in  other  parts  of  this  manual. 
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Figure  7-1  shows  how  the  functions  are  related  to  other  elements  of  the  E&V 
Reference  Manual  and  to  elements  in  the  E&V  Guidebook.  The  various  functions  that  are  useful 
in  defining  assessment  objectives  for  APSEs  or  APSE  components  are  described  in  this  chapter. 
The  relationships  between  tools  and  functions  given  in  this  chapter  are  typical  (or  traditional) 
relationships.  The  real  capabilities  should  be  determined  bv  the  tool  specifications  marketing 
claims,  or  the  like- 
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7.1  TRANSFORMATION 


Description: 

Transformation  features  describe  how  the  subject  is  manipulated  to  accommodate  the  user’s 
needs.  They  describe  what  transformations  take  place  as  input  to  the  tool  is  processed. 
[@Kean  1985] 


7.1.1  Editing 


Description: 

Selective  revision  of  computer-resident  data.  [@Kean  1985] 
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7.1.1 .1  Text  Editing 


Description: 

Editing  capabilities  provided  for  textual  data.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 

[Global 

4.12.2] 

Tools: 

[Word  Processor  5.8.2, 

Definition  Language  Processor  5.9.1, 

Specification  Language  Processor  5.9.2, 

Data  Model/Dictionary  5.9.4, 

Prototyping  Tools  5.9.7, 

Syntax-Directed  (Language-Sensitive)  Editor  5.11.5, 

Translating  Editor  5.12.1] 


Guidebook  References: 

[Completeness  6.4.9, 

@GB:  Text  Editing  Capabilities  Checklist 
Completeness  6.4.9, 

@GB:  Language-Sensitive  Editing  Capabilities  Checklist 
Power  6.4.21 , 

@GB:  Text  Editing  Capabilities  Checklist 
Power  6.4.21, 

@GB:  Language-Sensitive  Editing  Capabilities  Checklist 


l 


99.1; 

99.2; 

99.1; 

99.2] 
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7.1. 1.2  Data  Editing 


Editing  capabilities  provided  for  data  in  internal  (machine)  format. 


Life  Cycle  Activities: 

[Detailed  Design 
Coding  and  CSU  Testing 
CSC  Integration  and  Testing 
CSCI  Testing 

System  Integration  and  Testing 
Operational  Testing  and  Evaluation 


Tools: 

[Test  Description  and  Preparation  Services 


4.5.2, 

4.6.2, 

4.7.2,  4.7.3, 

4.8.2, 

4.9.2, 
4.10.2] 


5.14.4] 
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7.1. 1.3  Graphics  Editing 


Description: 

Editing  capabilities  provided  for  graphical  data. 


Cross  References: 


Life  Cycle  Activities: 
[Global 


Tools: 

[Graphics  Generator 
Enterprise  Model 
Data  Model/Dictionary 
Process  Model 
Prototyping  Tools 


[@Kean  1985] 


4.12.2] 


5.8.4, 

5.9.3, 

5.9.4, 

5.9.5, 
5.9.7] 
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7.1.2  Formatting 
Description: 

Arranging  data  according  to  predefined  and/or  user-defined  conventions.  [@Kean  1985] 


7.1. 2.1  MIL-STD  Format 

Description: 

The  formatting  of  documents  as  specified  by  the  MIL-STD(s). 


Cross  References: 


Life  Cycle  Activities: 

[Global  4.12.2] 


Tools: 

[Formatter  5.8.5] 
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7,1. 2.2  Table  of  Contents 


DfiggripiLon: 

The  production  of  a  table  of  contents  by  examining  the  contents  to  be  placed  within  the 
document. 


Cross  References: 


Life  Cycle  Activities: 
[Global 


4.12.2] 


Tools: 

[Word  Processor  5.8.2, 

Formatter  5.8.5] 


1 

1 
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1 

7.1. 2.3  Predefined  and  User-Defined  Forms 

1 

Description: 

9 

| 

The  ability  to  call  up  or  define  standard  formats  such 
D0D-STD-2167A. 

as  those  that  are  mentioned 

■ 

■ 

Cross  References: 

Life  Cycle  Activities: 

i 

[System  Requirements  Analysis/Design 

4.2.1  -  4.2.3,  4.2.6, 

Software  Requirements  Analysis 

4.3.1  -  4.3.3,  4.3.6, 

Preliminary  Design 

4.4.1  -  4.4.3,  4.4.6, 

Detailed  Design 

4.5.1  -  4.5.3,  4.5.6, 

Coding  and  CSU  Testing 

4.6.1,  4.6.2,  4.6.6, 

1 

* 

CSC  Integration  and  Testing 

4.7.1  -  4.7.3,  4.7.6. 

CSCI  Testing 

4.8.1  -  4.8.3,  4.8.5.  4.8.6, 

System  Integration  and  Testing 

4.9.2,  4.9.5, 

| 

Operation  Testing  and  Evaluation 

4.10.2, 

1 

Change  Requirements 

4.11.2,  4.11.5] 

9 

Tools: 

[Spreadsheet 

5.6.1, 

| 

Word  Processor 

5.8.2, 

V 

Formatter 

5.8.5, 

Applications  (Code)  Generator 

5.11.1, 

1 

Database  Schema  Generator 

5.11.2, 

Report  Generator 

5.11.3, 

Screen  Generator  5.11.4, 

if  Syntax-Directed  (Language-Sensitive)  Editor  5.11.5] 
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7.1.3  On-Line  Assistance  Processing 


Description: 

User  interface  that  is  part  of  the  input/output  process  of  a  programming  support  environment 
(e.g.,  command  assistance,  assistance,  on-line  tutoring,  definition  assistance,  etc.). 
[@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 

[Global  4.12.2] 

Tools: 

[On-Line  Assistance  5.1.7] 


7.1.4  Sort/Merge 
Description; 

The  process  of  arranging  data  in  a  specific  order  (e.g.,  alphabetical  order).  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 

[Global 

4.12.2] 

Tools: 

[Database  Manager 

5.2.2, 

Address/Phone  Book 

5.6.3, 

Electronic  Mail 

5.6.4, 

Dictionary/Thesaurus 

5.6.7] 
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7.1.5  Graphics  Generation 


Description: 

The  input,  construction,  storage,  retrieval,  manipulation,  alteration,  and  analysis  of  pictorial 
data  (e.g.,  generation  of  system  architectures,  software  designs,  financial  analysis,  maps, 
graphs,  etc.).  [@Kean  1985] 

Cross  References: 


Life  Cycle  Activities: 

[Global 

4.12.2] 

Tools: 

[Graphics  Generator 

5.8.4, 

Enterprise  Model 

5.9.3, 

Data  Model/Dictionary 

5.9.4, 

Process  Model 

5.9.5, 

Simulators 

5.9.6, 

Prototyping  Tools 

5.9.7, 

Screen  Generator 

5.11.4] 

7.1.6  Translation 


Description: 

The  conversion  from  one  language  form  to  another.  [@Kean  1985] 
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7.1. 6.1  Systems  Requirements 


Description: 

The  processing  of  formal  systems  requirements  statements  into  an  internal  database 
representation  for  subsequent  use.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 

[System  Requirements  Analysis/Design  4.2.2] 


Tools: 


[Definition  Language  Processor 

5.9.1, 

Enterprise  Model 

5.9.3, 

Process  Model 

5.9.5] 

Guidebook  References: 

[Augmentability 

6.4.4. 

@GB:  NADC/SPS  CASE  Tools  Evaluation 

9.4; 

Capacity 

6.4.6, 

@GB:  NADC/SPS  CASE  Tools  Evaluation 

9.4; 

Completeness 

6.4.9, 

@GB:  NADC/SPS  CASE  Tools  Evaluation 

9.4; 

Completeness 

6.4.9, 

@GB:  Time-Critical  Applications  Support  Checklist 

9.5; 

Processing  Effectiveness 

6.4.22, 

@GB:  NADC/SPS  CASE  Tools  Evaluation 

9.4; 

Self  Descriptiveness 

6.4.28, 

@GB:  NADC/SPS  CASE  Tools  Evaluation 

9.4; 

System  Clarity 

6.4.33, 

@GB:  NADC/SPS  CASE  Tools  Evaluation 

9.4; 

Training 

6.4.36, 

@GB:  NADC/SPS  CASE  Tools  Evaluation 

9.4] 
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7.1. 6.2  Software  Requirements 


Description: 

The  processing  of  formal  software  requirements  statements  into  an  internal  database 
representation  for  subsequent  use.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 

[Software  Requirements  Analysis  4.3.2] 


Tools: 

[Definition  Language  Processor  5.9.1, 

Data  Model/Dictionary  5.9.4, 

Process  Model  5.9.5] 


Guidebook  References: 

[Augmentability 

6.4.4, 

@GB:  NADC/SPS  CASE  Tools  Evaluation 

9.4; 

Capacity 

6.4.6, 

@GB:  NADC/SPS  CASE  Tools  Evaluation 

9.4; 

Completeness 

6.4.9, 

@GB:  NADC/SPS  CASE  Tools  Evaluation 

9.4; 

Completeness 

6.4.9. 

@GB:  Time-Critical  Applications  Support  Checklist 

9.5: 

Processing  Effectiveness 

6.4.22, 

@GB:  NADC/SPS  CASE  Tools  Evaluation 

9.4; 

Self  Descriptiveness 

6.4.28, 

@GB:  NADC/SPS  CASE  Tools  Evaluation 

9.4; 

System  Clarity 

6.4.33, 

@GB:  NADC/SPS  CASE  Tools  Evaluation 

9.4; 

Training 

6.4.36. 

@GB:  NADC/SPS  CASE  Tools  Evaluation 

9.4] 
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7.1. 6.3  Requirements  to  Natural  Language 


Description: 

The  transformation  of  formal  requirements  language  constructs  into  English  language  text. 
[@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 

[System  Requirements  Analysis/Design  4.2.2, 

Software  Requirements  Analysis  4.3.2] 


Tools: 
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7.1. 6.4  Preliminary  Design 


Description: 

The  processing  of  formal  preliminary  design  statements  into  an  internal  database 
representation  for  subsequent  use.  [@Kean  1985] 

Cross  References: 

Life  Cycle  Activities: 


[Preliminary  Design 

4.4.2] 

Tools: 

[Specification  Language  Processor 

5.9.2, 

Data  Model/Dictionary 

5.9.4, 

Process  Model 

5.9.5, 

Prototyping  Tools 

5.9.7] 

Guidebook  References: 

[Augmentability 

6.4.4. 

@GB:  NADC/SPS  CASE  Tools  Evaluation 

9.4; 

Capacity 

6.4.6, 

@GB:  NADC/SPS  CASE  Tools  Evaluation 

9.4; 

Completeness 

6.4.9, 

@GB:  NADC/SPS  CASE  Tools  Evaluation 

9.4; 

Completeness 

6.4.9, 

@GB:  Time-Critical  Applications  Support  Checklist 

9.5: 

Power 

6.4.21, 

@GB:  SEI  Design  Support  Experiment 

9.1; 

Processing  Effectiveness 

6.4.22, 

@GB:  SEI  Design  support  Experiment 

9.1; 

Processing  Effectiveness 

6.4.22, 

@GB:  NADC/SPS  CASE  Tools  Evaluation 

9.4; 

Self  Descriptiveness 

6.4.28, 

@GB:  NADC/SPS  CASE  Tools  Evaluation 

9.4: 

Storage  Effectiveness 

6.4.31, 

@GB:  SEI  Design  Supoort  Experiment 

9.1; 

System  Clarity 

6.4.33. 

@GB:  NADC/SPS  CASE  Tools  Evaluation 

9.4: 

Training 

6.4.36, 

@GB:  NADC/SPS  CASE  Tools  Evaluation 

9.4] 
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i 

7.1. 6.5  Detailed  Design 


Description: 

The  processing  of  formal  detailed  design  statements  into  an  internal  database  representation 
for  subsequent  use.  [@Kean  1985] 


Cross  References: 

Life  Cycle  Activities: 

[Detailed  Design  4.5.2] 


Tools: 

[Specification  Language  Processor 
Data  Model/Dictionary 
Process  Model 


5.9.2, 

5.9.4, 

5.9.5] 


Guidebook  References; 


[Completeness  6.4.9, 

@GB:  Time-Critical  Applications  Support  Checklist  9.5; 

Power  6.4.21, 

@GB:  SEI  Design  Support  Experiment  9.1; 

Processing  Effectiveness  6.4.22, 

@GB:  SEI  Design  Support  Experiment  9.1; 

Storage  Effectiveness  6.4.31, 

@GB:  SEI  Design  Support  Experiment  9.1] 
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7.1. 6.6  Assembling 


Description; 

Translating  a  program  expressed  in  an  assembly  language  into  object  code.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 


[Coding  and  CSU  Testing 

4.6.2, 

CSC  Integration  and  Testing 

4.7.2, 

CSCI  Testing 

4.8.2, 

System  Integration  and  Testing 

4.9.2, 

Operational  Testing  and  Evaluation 

4.10.2] 

Tools: 

[Host-Target  System  Cross-Assembler 


5.13.1] 


Guidebook  References: 

[Completeness  6.4.9, 

@GB:  Cross  Development  System  Support  Questionnaire  13.3: 
Power  6.4.21, 

@GB:  Assembling  Checklist  6.1] 
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7.1. 6.7  Compilation 


Description: 

Translating  a  computer  program  expressed  in  a  procedural  or  problem-oriented  language 
into  object  code.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing 

4.6.2, 

CSC  Integration  and  Testing 

4.7.2, 

CSCI  Testing 

4.8.2. 

System  Integration  and  Testing 

4.9.2. 

Operational  Testing  and  Evaluation 

4.10.2] 

Tools: 

[Translating  Editor 

5.12.1, 

Compiler  (Front  End) 

5.12.2, 

Compiler  (Code  Generator  —  Back  End) 

5.12.3] 

Guidebook  References: 


'[Anomaly  Management 

6.4.2, 

@GB:  ARTEWG  Catalogue  of  Ada  Runtime 

Implementation  Dependencies 

5.10; 

Anomaly  Management 

6.4.2, 

@GB:  Compiler  Assessment  Ouestionnaire 

5.12; 

Augmentability 

6.4.4, 

@GB:  Compiler  Assessment  Ouestionnaire 

5.12; 

Capacity 

6.4.6, 

@GB.  IDA  Benchmarks 

5.2: 

Capacity 

6.4.6, 

@GB:  MITRE  Benchmark  Generator  Tool 

5.6: 

Capacity 

6.4.6, 

@GB:  Compiler  Assessment  Ouestionnaire 

5.12: 

'NOTE:  Normally,  the  concern  of  evaluation  is  the  quality  of  a  tool.  However,  this 
evaluation  technique  focuses  on  the  product  of  the  tool  rather  than  the  tool  itself. 
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Commonality 

6.4.7, 

@GB:  Compiler  Assessment  Questionnaire 

5.12; 

Completeness 

6.4.9, 

@GB:  ACVC 

5.1; 

Completeness 

6.4.9, 

@GB:  Cross  Development  System  Support  Questionnaire 

13.3; 

Consistency 

6.4.10, 

@GB:  Compiler  Assessment  Questionnaire 

5.12; 

Cost 

6.4.11, 

@GB:  Compiler  Assessment  Questionnaire 

5.12; 

Document  Accessibility 

6.4.13, 

@GB:  Compiler  Assessment  Questionnaire 

5.12; 

Generality 

6.4.16. 

@GB:  Compiler  Assessment  Questionnaire 

5.12; 

Granularity 

6.4.17, 

@GB:  Compiler  Assessment  Questionnaire 

5.12; 

Maturity 

6.4.18, 

@GB:  Compiler  Assessment  Questionnaire 

5.12; 

Operability 

6.4.20. 

@GB:  Compiler  Assessment  Questionnaire 

5.12; 

Power 

6.4.21, 

@GB:  Compilation  Checklist 

5.8; 

Power 

6.4.21, 

@GB:  Compiler  Assessment  Questionnaire 

5.12; 

'Processing  Effectiveness 

6.4.22, 

@GB:  IDA  Benchmarks 

5.2; 

'Processing  Effectiveness 

6.4.22, 

@GB:  ACEC 

5.3; 

'Processing  Effectiveness 

6.4.22, 

@GB:  PIWG  Benchmark  Tests 

5.4; 

'Processing  Effectiveness 

6.4.22, 

@GB:  University  of  Michigan  Benchmark  Tests 

5.5; 

'Processing  Effectiveness 

6.4.22, 

@GB:  ARTEWG  Catalogue  of  Ada  Runtime 

Implementation  Dependencies 

5.10; 

Processing  Effectiveness 

6.4.22, 

@GB:  Compiler  Assessment  Questionnaire 

5.12; 

Proprietary  Rights 

6.4.23, 

@GB:  Compiler  Assessment  Questionnaire 

5.12; 

'NOTE:  Normally,  the  concern  of  evaluation  is  the  quality  of  a  tool.  However,  this 
evaluation  technique  focuses  on  the  product  of  the  tool  rather  than  the  tool  itself. 
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Rehostability 

6.4.25, 

@GB:  Compiler  Assessment  Questionnaire 

5.12; 

Required  Configuration 

6.4.26, 

@GB:  Compiler  Assessment  Questionnaire 

5.12; 

’Retargetability 

6.4.27, 

@GB:  ARTEWG  Catalogue  of  Ada  Runtime 

Implementation  Dependencies 

5.10: 

Retargetability 

6.4.27, 

@GB:  Compiler  Assessment  Questionnaire 

5.12; 

’Storage  Effectiveness 

6.4.31, 

@GB:  IDA  Benchmarks 

5.2: 

’Storage  Effectiveness 

6.4.31, 

@GB:  ACEC 

5.3: 

’Storage  Effectiveness 

6.4.31. 

@GB:  PIWG  Benchmark  Tests 

5.4: 

’Storage  Effectiveness 

6.4.31, 

@GB:  ARTEWG  Catalogue  of  Ada  Runtime 

Implementation  Dependencies 

5.10: 

Storage  Effectiveness 

6.4.31, 

@GB:  Compiler  Assessment  Questionnaire 

5.12; 

System  Compatibility 

6.4.34, 

@GB:  Compiler  Assessment  Questionnaire 

5.12: 

Training 

6.4.36, 

@GB:  Compiler  Assessment  Questionnaire 

5.12: 

Visibility 

6.4.38, 

@GB:  Compiler  Assessment  Questionnaire 

5.12] 

’NOTE:  Normally,  the  concern  of  evaluation  is  the  quality  of  a  tool.  However,  this 
evaluation  technique  focuses  on  the  product  of  the  tool  rather  than  the  tool  itself. 
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7.1. 6.8  Conversion 

Description: 

Modifying  an  existing  program  to  enable  it  to  operate  with  similar  functional  capabilities  in  a 
different  environment.  [@Kean  1985] 

Cross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing 

4.6.2, 

CSC  Integration  and  Testing 

4.7.2, 

CSCI  Testing 

4.8.2, 

System  Integration  and  Testing 

4.9.2, 

Operational  Testing  and  Evaluation 

4.10.2] 

Tools: 

[Host-Target  System  Cross-Assembler 

5.13.1] 

Guidebook  References; 
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7.1. 6.9  Macro  Expansion 


Description: 

Augmenting  instructions  in  a  source  language  with  user  defined  sequences  of  instructions  in 
the  same  source  language.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing  4.6.2, 

CSC  Integration  and  Testing  4.7.2, 

CSCI  Testing  4.8.2, 

System  Integration  and  Testing  4.9.2, 

Operational  Testing  and  Evaluation  4.10.2] 


Tools: 

[Applications  (Code)  Generator  5.11.1. 

Syntax-Directed  (Language-Sensitive)  Editor  5.11.5] 
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7.1.6.10  Structure  Preprocessing 
Description: 

Translating  a  computer  program  with  structured  constructs  into  its  equivalent  without 
structured  constructs.  [@Kean  1985] 


Cross  References: 

Life  Cycle  Activities: 

[Coding  and  CSU  Testing 

4.6.2, 

CSC  Integration  and  Testing 

4.7.2, 

CSCI  Testing 

4.8.2, 

System  Integration  and  Testing 

4.9.2, 

Operational  Testing  and  Evaluation 

4.10.2] 

Tools: 

[Database  Schema  Generator 

5.11.2] 
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7.1.6.11  Body  Stub  Generation 
Description: 

The  creation  of  null  bodies  for  specifications  requiring  bodies  whose  corresponding  bodies 
have  yet  to  be  defined.  [@Kean  1985] 

Cross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing  4.6.2, 

CSC  Integration  and  Testing  4.7.2] 


Tools: 

[Applications  (Code)  Generator  5.11.1, 

Test  Description  and  Preparation  Services  5.14.4] 


Guidebook  References: 
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7.1.6.12  Preamble  Generation 

Description,; 

Generating  the  preamble  for  a  main  program  that  contains  parameters.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 


[Coding  and  CSU  Testing 

4.6.2, 

CSC  Integration  and  Testing 

4.7.2, 

CSCI  Testing 

4.8.2, 

System  Integration  and  Testing 

4.9.2, 

Operational  Testing  and  Evaluation 

4.10.2] 

Tools: 

[Applications  (Code)  Generator 

5.11.1] 

QuidebooK  Bef  eiences; 
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7.1.6.13  Unking/Loading 


Description: 

The  creation  of  a  load/executable  module  on  the  host  machine  from  one  or  more 
independently  translated  object  modules  or  load  modules  by  resolving  cross-references 
among  the  object  modules,  and  possibly  relocating  elements.  [@Kean  1985] 

Cross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing  4.6.2, 

CSC  Integration  and  Testing  4.7.2, 

CSCI  Testing  4.8.2, 

System  Integration  and  Testing  4.9.2, 

Operational  Testing  and  Evaluation  4.10.2] 


Tools: 

[Host-Based  Target  Linker  5.13.2, 

Host-Based  Target  Loader  5.13.3] 


Guidebook  References: 


[Completeness 

6.4.9, 

@GB:  Cross  Development  System 

Support  Questionnaire 

13.3: 

Power 

6.4.21, 

@GB:  Linking/Loading  Checklist 

6.2] 
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7.1.6.14  Interpretation 
Description: 

The  translation  of  a  source  program  into  some  intermediate  data  structure,  then  executing 
the  algorithm  by  carrying  out  each  operation  given  in  the  intermediate  structure.  [@Kean 
1985] 

Cross  References: 


Life  Cycle  Activities: 


[Coding  and  CSU  Testing 

4.6.2, 

CSC  Integration  and  Testing 

4.7.2, 

CSCI  Testing 

4.8.2, 

System  Integration  and  Testing 

4.9.2, 

Operational  Testing  and  Evaluation 

4.10.2] 

Tools: 

[Interpreter 

5.12.4] 
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7.1.6.15  Software  and  System  Test  Communications 
Description: 

The  exchange  or  translation  of  information  or  data  from  one  level  of  test  for  use  in 
constructing  test  data  or  scenarios  for  other  levels  of  test.  For  example,  given  a  set  of  test 
data  from  a  unit  test,  constructing  system  test  inputs  to  drive  the  unit  in  an  identical  manner. 


Cross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing 
CSC  Integration  and  Testing 
CSCI  Testing 

System  Integration  and  Testing 
Operational  Testing  and  Evaluation 


4.6.2, 

4.7.2,  4.7.3, 

4.8.2,  4.8.3, 

4.9.2,  4.9.3, 

4.10.2,  4.10.3] 


Tools: 

[Test  Description  and  Preparation  Services  5.14.4] 
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7.1.6.16  Compression 
Description: 

The  translation  of  data  into  a  more  densely-packed  format  for  the  purposes  of  saving  space 
on  the  storage  media  and/or  reducing  transmission  time.  The  process  may  have  to  be 
reversed  in  order  for  the  data  to  be  processed  again. 


Cross  References: 


Life  Cycle  Activities: 
[Global 


4.12.2] 


Tools: 

[Archive.  Backup,  and  Retrieval  System 


5.1.2] 
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7.1.6.17  Expansion 


Description: 

The  translation  of  data  from  a  densely-packed  format  into  a  format  that  is  more  natural  for  the 
purpose  of  processing  the  data. 

Cross  References: 


Life  Cycle  Activities: 
[Global 


4.12.2] 


Tools: 

[Archive.  Backup,  and  Retrieval  System  5.1.2] 
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7.1.6.18  Encryption 


Description: 

The  translation  of  data  into  a  form  in  which  it  is  impossible  to  understand  the  meaning  of  the 
data  until  the  process  has  been  reversed. 

Cross  References: 


Life  Cycle  Activities: 

[Global  4.12.1] 


Tools: 

[Security  System  5.1.3] 


Guidebook  References; 
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7.1.6.19  Decryption 
Description: 

The  translation  of  data  from  a  form  in  which  it  is  impossible  to  understand  the  meaning  of  the 
data  into  a  form  in  which  the  data  can  be  understood. 

Cross  References: 


Life  Cycle  Activities: 

[Global  4  12.1] 


Tools: 

[Security  System  5.1.3] 
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7.1.7  Synthesis 


Description; 

The  generation  of  programs  according  to  predefined  rules  from  a  program  specification  or 
intermediate  language.  [@Kean  1985] 


7.1. 7.1  Design  Generation 
Description: 

The  translation  or  interpretation  used  to  construct  program  designs. 


Cross,  References: 


Life  Cycle  Activities: 

[Preliminary  Design  4.4.2, 

Detailed  Design  4.5.2] 


Tools: 


[Specification  Language  Processor 

5.9.2, 

Data  Model/Dictionary 

5.9.4, 

Process  Model 

5.9.5, 

Prototyping  Tools 

5.9.7] 
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7. 1.7.2  Requirements  Reconstruction 


The  extraction  ot  so, .ware  regents  statements  Iron,  a  pro9ram  0esl9n  ,an9oa9e. 


Life  Cycle  Activities: 

[Preliminary  Design 

Detailed  Design  4.4.4, 

CSC  Integration  and  Testing  4  5A' 


Tools: 

[Design  Library  Manager 
Reusable  Components  Library 
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7.1. 7.3  Program  Generation 


Description: 

The  translation  or  interpretation  used  to  construct  computer  programs  (e.g..  language 
translator  generator,  syntax  analyzer  generator,  code  generator  generator,  environment 
definition  generator,  user  interface  generator,  etc.).  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing  4.6.2, 

CSC  Integration  and  Testing  4.7.2,  4.7.3, 

CSCI  Testing  4.8.2,  4.8.3, 

System  Integration  and  Testing  4.9.2, 

Operational  Testing  and  Evaluation  4.10.2] 


Tools: 

[Prototyping  Tools  5.9.7, 

Applications  (Code)  Generator  5.11.1, 

Report  Generator  5.11.3, 

Screen  Generator  5.11.4, 

Tool  Building  Services  5.14.2] 
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7.1. 7.4  Source  Reconstruction 


Description: 

The  extraction  of  lexical  and  structure  attributes  from  an  intermediate  language.  [@Kean 
1985] 


Cross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing  4.6.4, 

CSC  Integration  and  Testing  4.7.4] 


Tools: 
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7.1. 7.5  Decompilation 


Description: 

The  translation  of  machine  code  sequences  into  a  higher  level  language  (i.e.,  back  into  the 
same  language  from  which  the  machine  code  sequences  were  generated).  [®Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing  4.6.4, 

CSC  Integration  and  Testing  4.7.4] 


Tools: 
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7.1. 7.6  Disassembling 


Description: 

The  translation  of  machine  code  sequences  into  assembly  language  sequences.  [@Kean 
1985] 


Cross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing  4.6.4. 

CSC  Integration  and  Testing  4.7.4] 


Tools: 
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7.1. 7.7  Test  Harness  Generation 

Pesoription; 

The  construction  of  an  environment  for  running  software  component  tests,  linking  test 
capabilities  into  an  integrated  toolset  to  perform  specific  tests,  accepting  program  inputs, 
simulating  missing  components,  comparing  actual  outputs  with  expected  outputs  to 
determine  correctness,  and  reporting  discrepancies.  [@DeMillo  1987] 


Cross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing  4.6.2, 

CSC  Integration  and  Testing  4.7.2,  4.7.3, 

CSCI  Testing  4.8.2,  4.8.3, 

System  Integration  and  Testing  4.9.2, 

Operational  Testing  and  Evaluation  4.10.2] 


Tools: 

[Test  Execution  Services  5.14.5] 
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7.2  MANAGEMENT 

Description: 

Features  that  aid  the  management  or  control  of  system/software  development.  [@Kean 
1985] 


7.2.1  Information  Management 
Description: 

The  organization,  accessing,  modification,  dissemination,  and  processing  of  information  that 
is  associated  with  the  development  of  a  software  system.  [@Kean  1985] 
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7.2.1. 1  Database  (Object)  Management 

Description: 

Managing  a  collection  of  interrelated  data  stored  together  with  controlled  redundancy  to 
serve  one  or  more  applications  and  independent  of  the  programs  using  the  data.  [@Kean 
1985] 

Cross  References: 


Life  Cycle  Activities: 

[Detailed  design  4.5.2, 

Coding  and  CSU  Testing  4.6.2, 

CSC  Integration  and  Testing  4.7.2,  4.7.3, 

CSCI  Testing  4.8.2,  4.8.3, 

System  Integration  and  Testing  4.9.2, 

Operational  Testing  and  Evaluation  4.10.2, 

Global  4.12.1] 


Tools: 

[Database  Manager  5.2.2, 

Virtual  Operating  System  5.3.1, 

Enterprise  Model  5.9.3, 

Data  Model/Dictionary  5.9.4, 

Process  Model  5.9.5, 

Design  Library  Manager  5.9.8, 

Data  Definition  Language  Processor  5.11.6, 

Data  Manipulation  Language  Processor  5.11.7, 

Reusable  Components  Library  5.11.8] 


Guidebook  References: 

[Completeness  6.4.9, 

@GB:  Database  Management  Checklist 

Power  6.4.21, 

@GB:  Database  Management  Checklist 


15.2; 

15.2] 
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7.2.1. 2  Documentation  Management 
Description: 

The  development  and  control  of  software  documentation.  [@Kean  1985] 

Cross  References: 

Life  Cycle  Activities: 

[System  Requirements  Analysis/Design  4.2.3, 

Software  Requirements  Analysis  4.3.3, 

Preliminary  Design  4.4.2,  4.4.3, 

Detailed  Design  4.5.2,  4.5.3, 

Coding  and  CSU  Testing  4.6.2, 

CSC  Integration  and  Testing  4.7.2,  4.7.3, 

CSCI  Testing  4.8.2,  4.8.3, 

System  Integration  and  Testing  4.9.2, 

Operational  Testing  and  Evaluation  4.10.2, 

Global  4.12.1] 


Tools: 

[Document  Manager  5.8.1] 
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7.2.1. 3  File  Management 


Description: 

Providing  and  controlling  access  to  files  associated  with  the  development  of  software. 
[@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 

[Detailed  Design 

4.5.2. 

Coding  and  CSU  Testing 

4.6.2, 

CSC  Integration  and  Testing 

4.7.2,  4.7.3, 

CSCI  Testing 

4.8.2,  4.8.3. 

System  Integration  and  Testing 

4.9.2, 

Operational  Testing  and  Evaluation 

4.10.2, 

Global 

4.12.1] 

Tools: 

[File  Manager 

5.2.1, 

Virtual  Operating  System 

5.3.1] 

Guidebook  References: 

[Completeness 

6.4.9, 

@GB:  File  Management  Checklist 

15.1; 

Power 

6.4.21. 

@GB:  File  Management  Checklist 

15.1] 
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7.2.1. 4  Electronic  Mail 


Description: 

The  process  of  receiving/sending  messages  from/to  other  system  users.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 

[Global 

4.12.1] 

Tools: 

[Address/Phone  Book 

5.6.3, 

Electronic  Mail 

5.6.4] 

Guidebook  References: 

(Completeness 

6.4.9, 

@GB:  Electronic  Mail  Checklist 

15.3 

Power 

6.4.21, 

@GB:  Electronic  Mail  Checklist 

15.3 
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7.2.1. 5  Electronic  Conferencing 


Description: 

The  concurrent  on-line  correspondence  between  two  or  more  users.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 
[Global 


4.12.1] 


Tools: 

[Address/Phone  Book  5.6.3, 

Electronic  Conferencing  5.6.5] 
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7.2.1. 6  Specification  Management 


Description: 

The  control  of  requirements  specifications.  Specification  management  features  are 
somewhat  methodology  dependent  because  associated  with  them  are  requirements 
languages  with  formal  procedures  for  their  use.  [@Kean  1985] 


Dross  References: 


Life  Cycle  Activities: 

[Software  Requirements  Analysis  4.3.5, 

Preliminary  Design  4.4.5, 

Detailed  Design  4.5.5] 


Tools: 

[Specification  Language  Processor  5.9.2, 

Data  Model/Dictionary  5.9.4, 

Process  Model  5.9.5] 
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7.2.1. 7  Program  Library  Management 


Description; 

The  creation,  manipulation,  display,  and  deletion  of  the  various  components  of  a  program 
library.  A  program  library  is  a  repository  for  all  program  information  (e.g.,  source  programs, 
object  programs,  executable  programs,  documentation,  data,  etc.).  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 

[Software  Requirements  Analysis  4.3.2, 

Preliminary  Design  4.4.2, 

Detailed  Design  4.5.2, 

Coding  and  CSU  Testing  4.6.2, 

CSC  Integration  and  Testing  4.7.2, 

CSCI  Testing  4.8.2. 

System  Integration  and  Testing  4.9.2, 

Operational  Testing  and  Evaluation  4.10.2] 


Tools: 

[Configuration  Control  5.7.2, 

Version  Control  5.7.4, 

Design  Library  Manager  5.9.8, 

Reusable  Components  Library  5.11.8, 

Program  Library  Manager  5.12.5] 


Guidebook  References; 

[Completeness  6.4.9, 

@GB:  Program  Library  Management  Checklist  5.9: 

Power  6.4.21, 

@GB:  Program  Library  Management  Checklist  5.9] 
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7.2.1. 8  Test  Data  Management 


Description: 

The  development  and  control  of  software  test  data.  [@Kean  1985] 

Cross  References: 


Life  Cycle  Activities: 

[Detailed  Design 

4.5.2. 

Coding  and  CSU  Testing 

4.6.2, 

CSC  Integration  and  Testing 

4.7.2,  4.7.3, 

CSCI  Testing 

4.8.2,  4.8.3, 

System  Integration  and  Testing 

4.9.2, 

Operational  Testing  and  Evaluation 

4.10.2] 

Tools: 

[Test  Description  and  Preparation  Services 


5.14.4] 
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7.2.1. 9  Evaluation  Results  Management 


The  cataloguing  and  maintenance  of  the  results  from  the  operational  test  and  evaluation  of 
the  product.  [@Kean  1985] 


Life  Cycle  Activities: 

[Operational  Testing  and  Evaluation 


Tools: 

[Test  Analysis  Services 


4.10.5] 


5.14.6] 
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7.2.1.10  Performance  Monitoring 


Description: 

The  monitoring  of  the  performance  characteristics  of  the  finished  product.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 

[Global  4.12.1] 


Tools: 

[Performance  Monitor  5.1.9] 


Guidebook  References: 

[Completeness  6.4.9, 

@GB:  Performance  Monitoring  Checklist  99.3] 
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7.2.1.11  Data  and  Error  Logging 
Description: 

The  automatic  capture  of  information  describing  system  state  changes  and  anomalies  or 
exceptions  that  occur  during  test  execution. 

Cross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing 
CSC  Integration  and  Testing 
CSCI  Testing 

System  Integration  and  Testing 
Operational  Testing  and  Evaluation 


4.6.2, 

4.7.2,  4.7.3, 

4.8.2,  4.8.3, 

4.9.2,  4.9.3, 

4.10.2,  4.10.3] 


Tools: 

[Test  Execution  Services 


5.14.5] 
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7.2.1.12  Status  Display 
Description: 

The  presentation  of  information  concerning  test  execution,  software  operation,  and  system 
state. 

Cross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing  4.6.2, 

CSC  Integration  and  Testing  4.7.2,  4.7.3, 

CSCI  Testing  4.8.2,  4.8.3, 

System  Integration  and  Testing  4.9.2, 

Operational  Testing  and  Evaluation  4.10.2] 


Tools: 

[Test  Analysis  Services  5.14.6] 
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7.2.2  Project  Management 
Description: 

The  management  of  a  system/software  development  project.  [@Kean  1985] 


7.2.2.1  Cost  Estimation 


Description: 

The  process  of  determining  the  amount  of  labor  necessary  for  the  completion  of  a  task,  the 
amount  and  potential  costs  of  computer  time  required,  etc.,  prior  to  and  during  a  project’s 
lifetime.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 
[Global 


4.12.1] 


Tools: 

[Cost  Estimator 

Ggidebook  eeftrence-S; 


5.5.1] 
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7. 2.2.2  Quality  Specification 


Description: 

The  selection  and  prioritization  of  quality  factors  required  for  a  given  application:  availability 
of  techniques  to  apply  trade-off  analyses  (quality  factor  vs.  quality  factor,  and  quality  factor 
vs.  cost)  and  quantify  quality  requirements.  Potential  quality  factors  include:  efficiency, 
integrity,  reliability,  survivability,  usability,  correctness,  maintainability,  verifiability/testability, 
expandability/flexibility,  interoperability,  reusability,  and  transportability.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 
[Global 


4.12.1] 


Tools: 
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7.2.2.3  Scheduling 


Description: 

The  process  of  identifying  tasks,  products  to  be  delivered,  delivery  dates,  personnel  needed 
to  complete  tasks,  etc.  for  a  development  project.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 
[Global 


4.12.1] 


Tools: 

[Scheduler  5.5.3, 

Calendar  5.6.6] 
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7.2.2.4  Work  Breakdown  Structure 


Description: 

The  enumeration  of  all  work  activities  in  hierarchic  refinements  of  detail  that  divides  work  to 
be  done  into  short,  manageable  tasks  with  quantifiable  inputs,  outputs,  schedules,  and 
assigned  responsibilities.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 

[Global  4.12.1] 


Tools: 

[Work  Breakdown  Structure  Editor 


5.5.4] 
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7.2.2. 5  Resource  Estimation 


Bascrigtioni 

The  estimation  of  resources  attributed  to  an  entity.  (@Kean  1985) 


£rg>S5JBeferenc?<f 


4.3.1, 

4.4.1, 

4.5.1, 

4.6.1, 

4.7.1, 
4.8.1] 


5.5.5] 

^irigfcbgQk  Reference?' 


uire  ^ycie  Activities: 

[Software  Requirements  Analysis 
Preliminary  Design 
Detailed  Design 
Coding  and  CSU  Testing 

clc,Tes9;TnandTeS,in9 


Tools: 

[Resource  Estimator 
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7.2.2.6  Tracking 


Description: 

The  tracking  of  the  development  of  an  entity  through  the  software  life  cycle.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 

[Software  Requirements  Analysis  4.3.1, 

Preliminary  Design  4.4.1, 

Detailed  Design  4.5.1, 

Coding  and  CSU  Testing  4.6.1, 

CSC  Integration  and  Testing  4.7.1, 

CSCI  Testing  4.8.1] 


Tools: 

[Tracking  5.5.6, 

Analysis  and  Reporting  5.5.9, 

Configuration  Status  Accounting  5.7.3, 

History  5.7.5] 
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7.2.2.7  Configuration  Management 
Description: 

The  establishment  of  baselines  for  configuration  items,  the  control  of  changes  to  these 
baselines,  and  the  control  of  releases  to  the  operational  environment.  [@Kean  1985] 

Cross  References: 


Life  Cycle  Activities: 

[Global 

4.12.5] 

Tools: 

[Configuration  Identification 

5.7.1, 

Configuration  Control 

5.7.2, 

Configuration  Status  Accounting 

5.7.3, 

Version  Control 

5.7.4, 

History 

5.7.5] 

Guidebook  References: 


[Augmentability 

@GB:  Configuration 
Capacity 

@GB:  Configuration 
Commonality 

@GB:  Configuration 
Completeness 

@GB:  Configuration 
Completeness 

@GB:  Configuration 
Consistency 

@GB:  Configuration 

Cost 

@GB:  Configuration 
Document  Accessibility 

@GB:  Configuration 
Generality 

@GB:  Configuration 


6.4.4, 

Management  Assessment  Questionnaire 

6.4.6, 

Management  Assessment  Questionnaire 

6.4.7. 

Management  Assessment  Questionnaire 

6.4.9, 

Management  Capabilities  Checklist 

6.4.9, 

Management  Assessment  Questionnaire 

6.4.10, 

Management  Assessment  Questionnaire 

6.4.11, 

Management  Assessment  Questionnaire 

6.4.13, 

Management  Assessment  Questionnaire 

6.4.16, 

Management  Assessment  Questionnaire 


10.3 

10.3 

10.3 

10.1 

10.3 

10.3 

10.3 

10.3 

10.3 
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Granularity 

@GB: 

Maturity 

@GB: 

Operability 

@GB: 

Power 

@GB: 

Power 


6.4.17, 

Configuration  Management  Assessment  Questionnaire 

6.4.18, 

Configuration  Management  Assessment  Questionnaire 

6.4.20, 

Configuration  Management  Assessment  Questionnaire 

6.4.21, 

SEI  Configuration  Management  Experiment 

6.4.21, 


@GB:  Configuration  Management  Assessment  Questionnaire 
Processing  Effectiveness  6.4.22, 

@GB:  SEI  Configuration  Management  Experiment 


Processing  Effectiveness 


6.4.22. 


@GB:  Configuration  Management  Assessment  Questionnaire 


Proprietary  Rights 


6.4.23, 


@GB:  Configuration  Management  Assessment  Questionnaire 


Rehostability 
@GB 

Storage  Effectiveness 


6.4.25, 

Configuration  Management  Assessment  Questionnaire 

6.4.31, 


@GB:  Configuration  Management  Assessment  Questionnaire 


System  Compatibility 


6.4.34, 


Training 


@GB:  Configuration  Management  Assessment  Questionnaire 


6.4.36, 


@GB:  Configuration  Management  Assessment  Questionnaire 


Visibility 


6.4.38, 


@GB:  Configuration  Management  Assessment  Questionnaire 


10.3; 

10.3; 

10.3; 

10.2; 

10.3; 

10.2; 

10.3; 

10.3; 

10.3; 

10.3; 

10.3; 

10.3; 

10.3] 
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7.2.2.8  Quality  Assessment 

Description; 

The  use  of  field  data  to  determine  the  achieved  level  of  quality  in  deployed  software  systems. 
A  verification  that  the  specified  quality  requirements  have  been  met.  [@Kean  1985] 

Cross  References: 

Life  Cycle  Activities: 

Tools: 

[Design  Library  Manager 
Reusable  Components  Library 

Guidebook  References; 


5.9.8, 

5.11.8] 
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7.2.2.9  Risk  Analysis 


Description: 

The  process  of  determining  the  amount  of  risk  associated  with  the  completion  of  a  task,  prior 
to  and  during  a  project's  lifetime.  Risk  may  be  defined  as  the  probability  of  missing  a 
deadline,  probability  of  overrunning  a  budget,  probability  of  technological  failure,  and  the  like. 
Also  includes  the  analysis  of  the  operational  system,  for  example,  mean  time  between 
failures  (MTBF)  and  number  of  remaining  errors. 


Cross  References: 


Life  Cycle  Activities: 

[Global  4.12.1] 


Tools: 

[Analysis  and  Reporting  5.5.9] 
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7.2.3  Computer  System  Management 


Description: 

The  management  of  hardware/software  architectures  to  support  the  life  cycle  software 
engineering  environment.  Such  services  include:  creating,  scheduling,  and  removing  tasks; 
switching  the  processor  among  tasks;  sending  messages  between  tasks;  providing  direct 
and  import/export  access;  managing  distributed  systems;  sending  files  from  one  host 
machine  to  another  on  a  network,  etc.  [@Kean  1985] 


7-63 


E&V  Reference  Manual,  Version  2.0 


7.2.3.1  Command  Language  Processing 


Description: 

The  processing  of  command  language  constructs  into  functions  performed  by  the  operating 
system.  [@Kean  1985] 

Cross  References: 


Life  Cycle  Activities: 

[Global  4.12.2] 

Tools: 

[Command  Language  Interface  5.1.1] 

Guidebook  References: 


[Anomaly  Management 

6.4.2, 

Augmentability 

6.4.4, 

Capacity 

6.4.6, 

Commonality 

6.4.7, 

Consistency 

6.4.10, 

Cost 

6.4.11, 

Document  Accessibility 

6.4.13, 

Generality 

6.4.16, 

Granularity 

6.4.17, 

Maturity 

6.4.18, 

Operability 

6.4.20, 

Power 

6.4.21, 

Processing  Effectiveness 

6.4.22, 

Proprietary  Rights 

6.4.23, 

Rehostability 

6.4.25, 

Required  Configuration 

6.4.26, 

Storage  Effectiveness 

6.4.31, 

System  Compatibility 

6.4.34, 

Training 

6.4.36, 

Visibility 

6.4.38, 

@GB:  Command  Language  Interpreter 


Assessment  Questionnaire  99.4] 
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7.2.3.2  Input/Output  Support 


Description: 

The  services  for  accessing  standard  I/O  devices  such  as  disks,  tapes,  terminals,  and 
printers  from  within  a  program. 


Cross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing  4.6.2, 

CSC  Integration  and  Testing  4.7.2,  4.7.3, 

CSCI  Testing  4.8.2,  4.8.3, 

System  Integration  and  Testing  4.9.2, 

Operational  Testing  and  Evaluation  4.10.2, 

Global  4.12.2] 


Tools: 

[Input  and  Output  Services  5.1.8, 

Virtual  Operating  System  5.3.1, 

Window  Manager  5.3.2, 

I/O  Pipes  5.3.4, 

RAM  Cache  5.3.5, 

Runtime  System  5.12.6] 
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7.2.3.3  Kernel 


Description: 

The  functions  which  provide  access  for  tools  and  application  programs  to  services  of  the 
native  operating  system. 

Cross  References: 


Life  Cycle  Activities: 

[Global  4.12.2] 


Tools: 

[Security  System  5.1.3, 

Job  Scheduler  5.1.4, 

Resource  Controller  5.1.5. 

Virtual  Operating  System  5.3.1, 

Window  Manager  5.3.2] 


Guidebook  References: 

[Completeness  6.4.9, 

@GB:  CIVC  8.1; 

Completeness  6.4.9, 

@GB:  Tool  Support  Interface  Evaluation  8.2] 
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7.2.3.4  Math/Statistics 


Description: 

The  services  for  supporting  standard  math  and  statistical  operations. 


Cross  References: 


Life  Cycle  Activities: 
[Global 


4.12.2] 


Tools: 

[Spreadsheet  5.6.1, 

Calculator  5.6.2] 
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7.2.3.5  Runtime  Environment 


Description: 

The  functions  that  can  be  expected  to  be  found  in  the  runtime  libraries  for  Ada 
implementations.  “It  should  be  noted  that  the  dividing  line  between  the  predefined  runtime 
support  library  on  the  one  hand  and  the  conventions  and  data  structures  of  a  compiler  on  the 
other  hand  is  not  always  obvious.  One  Ada  implementation  may  use  a  predefined  routine  to 
implement  a  particular  language  feature,  while  another  implementation  may  realize  the  same 
feature  through  conventions  for  the  executable  code.  Not  addressed  here  are  elements  of  a 
runtime  library  that  exist  because  of  special  characteristics  of  the  underlying  computing 
resource.”  [@ARTEWG  1988] 


Cross  References; 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing 
CSC  Integration  and  Testing 
CSCI  Testing 

System  Integration  and  Testing 
Operational  Testing  and  Evaluation 


4.6.2, 

4.7.2,  4.7.3, 

4.8.2,  4.8.3, 

4.9.2,  4.9.3, 

4.10.2,  4.10.3] 


Tools: 

[Job  Scheduler  5.1.4, 

Resource  Controller  5.1.5, 

Virtual  Operating  System  5.3.1, 

Runtime  System  5.12.6] 


[Anomaly  Management  6.4.2, 

@GB:  Runtime  Support  System  Questionnaire  5.14 

Communication  Effectiveness  6.4.8, 

@GB:  Runtime  Support  System  Questionnaire  5.14 

Completeness  6.4.9, 

@GB:  ARTEWG  Runtime  Environment  Taxonomy  5.11 

Completeness  6.4.9, 

@GB:  Runtime  Support  System  Questionnaire  5.14 

Functional  Scope  6.4.15, 

@GB:  Runtime  Support  System  Questionnaire  5.14 
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Generality  6.4.16, 

@GB:  Runtime  Support  System  Questionnaire  5.14; 

Granularity  6.4.17, 

@GB:  Runtime  Support  System  Questionnaire  5.14; 

Modularity  6.4.19, 

(g*3B:  Runtime  Support  System  Questionnaire  5.14; 

Retargetability  6.4.27, 

@GB:  Runtime  Support  System  Questionnaire  5.14; 

System  Accessibility  6.4.32, 

@GB:  Runtime  Support  System  Questionnaire  5.14; 

System  Compatibility  6.4.34, 

@GB:  Runtime  Support  System  Questionnaire  5.14] 
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7.2.3.6  Import/Export 


Description: 

The  services  for  communicating  objects  between  various  computer  systems  or  networks. 


Cross  References: 


Life  Cycle  Activities: 

[CSCI  Testing  4.8.2, 

System  Integration  and  Testing  4.9.2, 

Operational  Testing  and  Evaluation  4.10.2. 

Global  4.12.2] 


Tools: 

[Archive,  Backup,  and  Retrieval  System  5.1.2, 

Import/Export  System  5.1.6, 

Virtual  Operating  System  5.3.1, 

Host-to-Target  Downloader  5.13.7, 

Target-to-Host  Uploader  5.13.8] 


Guidebook  References: 


[Completeness 

@GB:  Import/Export  Checklist 


6.4.9, 


6.3] 
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7.2.3.7  Access  Control 


Description: 

The  process  of  controlling  access  to  various  components  of  the  environment. 


Cross  References: 


Life  Cycle  Activities: 
[Global 


4.12.1] 


Tools: 

[Security  System  5.1.3, 

Virtual  Operating  System  5.3.1] 
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7.2.3.8  Job  Scheduling 


Description: 

The  process  of  controlling  the  execution  of  the  programs. 


Cross  References: 


Life  Cycle  Activities: 
[Global 


4.12.2] 


Tools: 

[Job  Scheduler  5.1.4, 

Virtual  Operating  System  5.3.1] 
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7.2.3.9  Resource  Management 
Description: 

The  process  of  managing  computing  resources  such  as  main  and  secondary  memory  and 
I/O  channels  to  processes  on  the  system. 

Cross  References: 


Life  Cycle  Activities: 
[Global 


4.12.2] 


Tools: 

[Resource  Controller  5.1.5, 

Virtual  Operating  System  5.3.1] 
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7.3  ANALYSIS 
Description: 

The  features  that  provide  an  examination  of  a  substantial  whole  to  determine  both  qualitative 
and  quantitative  properties.  [@Kean  1985] 


Guidebook  References: 


[Completeness 

6.4.9, 

@GB:  Testing  Capabilities  Checklist 

7.1; 

Power 

6.4.21, 

@GB:  Testing  Capabilities  Checklist 

7.1] 

7.3.1  Static  Analysis 

Description: 

Static  analysis  features  specify  operations  on  the  subject  without  regard  to  the  executability 
of  the  subject.  They  describe  the  manner  in  which  the  subject  is  analyzed.  [@Kean  1985] 

7.3. 1.1  Comparison 

Description: 

The  determination  and  assessment  of  similarities  between  two  or  more  items.  [@Kean  1 985] 

Cross  References: 

Life  Cycle  Activities: 

[Coding  and  CSU  Testing 

4.6.2,  4.6.4, 

CSC  Integration  and  Testing 

4.7.2  -  4.7.4, 

CSCI  Testing 

4.8.2,  4.8.3, 

System  Integration  and  Testing 

4.9.2, 

Operational  Testing  and  Evaluation 

4.10.2, 

Global 

4.12.4] 

Tools: 

[Dictionary/Thesaurus 

5.6.7, 

Test  Description  and  Preparation  Services 

5.14.4] 
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7.3. 1.2  Spelling  Checking 
Description: 

The  identification  of  incorrectly  spelled  words.  [@Kean  1985] 

Cross  References: 


Life  Cycle  Activities: 
[Global 


4.12.4] 


Tools: 

[Dictionary/Thesaurus  5.6.7, 

Spell  Checker  5.8.3] 


Guidebook  References: 
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7.3. 1.3  Data  Flow  Analysis 


Description: 

The  analysis  of  data  transformation  paths  to  identify  anomalies  and  specify  paths  to  be 
exercised  during  testing.  [@DeMillo  1987] 


Cross  References: 


Life  Cycle  Activities: 


[Software  Requirements  Analysis 

4.3.2, 

CSC  Integration  and  Testing 

4.7.2, 

CSCI  Testing 

4.8.2, 

System  Integration  and  Testing 

4.9.2, 

Operational  Testing  and  Evaluation 

4.10.2] 

Tools: 

[Static  Analyzers 

5.14.1] 

Guidebook  References: 
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7.3. 1.4  Functional  Analysis 

Description! 

The  analysis  of  formally  stated  requirements  to  determine  their  consistency  and 
completeness.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 

[Software  Requirements  Analysis  4.3.2] 


Tools: 

[Static  Analyzers  5.14.1] 


Guidebook  .References: 
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7.3.1. 5  Interface  Analysis 


Description: 

The  checking  of  interfaces  between  program  elements  for  consistency  and  adherence  to 
predefined  rules  and/or  axioms.  [@Kean  1985] 


QrpSS 


References: 


Life  Cycle  Activities: 

[Preliminary  Design  4.4.2, 

Detailed  Design  4.5.2, 

Coding  and  CSU  Testing  4.6.2, 

CSC  Integration  and  Testing  4.7.2, 

CSCI  Testing  4.8.2, 

System  Integration  and  Testing  4.9.2, 

Operational  Testing  and  Evaluation  4.10 .2] 


Tools: 

[Process  Model  5.9.5, 

Design  Library  Manager  5.9.8, 

Applications  (Code)  Generator  5.11.1, 

Reusable  Components  Library  5.11.8, 

Static  Analyzers  5.14.1] 
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7.3.1. 6  Traceability  Analysis 


Description: 

The  checking  for  internal  consistency  within  the  software  requirements  specifications. 
[@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 

[Software  Requirements  Analysis  4.3.2] 


Tools: 

[Data  Model/Dictionary  5.9.4, 

Process  Model  5.9.5, 

Static  Analyzers  5.14.1] 


E&V  Reference  Manual,  Version  2.0 


7.3.1. 7  Testability  Analysis 


Description: 

The  quantitative  measurement  of  the  extent  to  which  a  component  facilitates  the 
establishment  of  verification  criteria  and  supports  evaluation  of  its  performance. 


Cross  References: 


Life  Cycle  Activities: 


[System  Requirements  Analysis/Design 

4.2.3, 

Software  Requirements  Analysis 

4.3.4, 

Preliminary  Design 

4.4.2, 

Detailed  Design 

4.5.2, 

Coding  and  CSU  Testing 

4.6.2, 

CSC  Integration  and  Testing 

4.7.2, 

CSCI  Testing 

4.8.2, 

System  Integration  and  Testing 

4.9.2, 

Operational  Testing  and  Evaluation 

4.10.2] 

Tools: 

[Static  Analyzers 

5.14.1] 

Guidebook  References: 
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7.3.1. 8  Test  Condition  Analysis 

Description: 

The  analysis  of  formal  requirements  language  statements  to  determine  data  values  to  be 
examined  and  mechanisms  to  be  used  in  the  verification  of  test  results.  [@Kean  1985] 

Cross  References: 


Life  Cycle  Activities: 


[Preliminary  Design 

4.4.3. 

Detailed  Design 

4.5.2,  4.5.3, 

Coding  and  CSU  Testing 

4.6.2, 

CSC  Integration  and  Testing 

4.7.2,  4.7.3, 

CSCI  Testing 

4.8.2, 

System  Integration  and  Testing 

4.9.2, 

Operational  Testing  and  Evaluation 

4.10.2] 

Tools: 

[Test  Building  Services  5.14.3] 


Guidebook  References: 
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7.3.1. 9  Quality  Measurement 


Description: 

The  quantitative  measurement  of  specified  quality  factors  for  use  during  the  evaluation  of 
software  products  (and  prediction  of  software  quality)  at  key  milestones  during  development. 
Factors  to  be  analyzed  include:  efficiency,  integrity,  reliability,  survivability,  usability, 
correctness,  maintainability,  verifiability/testability,  expandability/flexibility,  interoperability, 
reusability,  and  transportability.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 


[System  Requirements  Analysis/Design 

4.2.2, 

Software  Requirements  Analysis 

4.3.2, 

Preliminary  Design 

4.4.2, 

Detailed  Design 

4.5.2, 

Coding  and  CSU  Testing 

4.6.2, 

CSC  Integration  and  Testing 

4.7.2, 

CSCI  Testing 

4.8.2, 

System  Integration  and  Testing 

4.9.2, 

Operational  Testing  and  Evaluation 

4.10.2] 

Tools: 

[Static  Analyzers 

5.14.1, 

Test  Execution  Services 

5.14.5] 
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7.3.1.10  Complexity  Measurement 


Description: 

The  determination  of  how  complicated  an  entity  (e.g.,  routine,  program,  system,  etc.)  is  by 
evaluating  some  number  of  associated  characteristics.  [@Kean  1985] 


Cross  References: 

Life  Cycle  Activities: 

[Preliminary  Design 

4.4.2, 

Detailed  Design 

4.5.2, 

Coding  and  CSU  Testing 

4.6.2, 

CSC  Integration  and  Testing 

4.7.2. 

CSCI  Testing 

4.8.2, 

System  Integration  and  Testing 

4.9.2, 

Operational  Testing  and  Evaluation 

4.10.2] 

Tools: 

[Static  Analyzers 


5.14.1] 
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7.3.1.11  Correctness  Checking 


Description: 

The  determination  of  agreement  between  the  component  as  coded  and  the  programming 
specification  (algorithmic  correctness). 


Cross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing  4.6.2, 

CSC  Integration  and  Testing  4.7.2,  4.7.3, 

CSCI  Testing  4.8.2.  4.8.3, 

System  Integration  and  Testing  4.9.2, 

Operational  Testing  and  Evaluation  4.10.2] 


Tools: 

[Data  Model/Dictionary  5.9.4, 

Process  Model  5.9.5, 

Test  Analysis  Services  5.14.6] 
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7.3.1.12  Completeness  Checking 


Description: 

The  assessment  of  whether  or  not  an  entity  has  all  its  parts  present  and  if  those  parts  are  fully 
developed.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 

[System  Requirements  Analysis/Design 
Software  Requirements  Analysis 
Preliminary  Design 
Detailed  Design 
Coding  and  CSU  Testing 
CSC  Integration  and  Testing 
CSCI  Testing 

System  Integration  and  Testing 
•  Operational  Testing  and  Evaluation 


4.2.2, 

4.3.4, 

4.4.4, 

4.5.4, 

4.6.2,  4.6.4, 

4.7.2,  4.7.4, 
4.8.2  -  4.8.4, 

4.9.2,  4.9.4, 

4.10.2,  4.10.4] 


Tools: 

[Data  Model/Dictionary 
Process  Model 
Static  Analyzers 


5.9.4, 

5.9.5, 
5.14.1] 
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7.3.1.13  Consistency  Checking 


Description: 

The  determination  of  whether  or  not  an  entity  is  internally  consistent  in  the  sense  that  it  is 
consistent  with  its  specification.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 

[System  Requirements  Analysis/Design 
Software  Requirements  Analysis 
Preliminary  Design 
Detailed  Design 
Coding  and  CSU  Testing 
CSC  Integration  and  Testing 
CSCI  Testing 

System  Integration  and  Testing 
Operational  Testing  and  Evaluation 


4.2.2  -  4.2.4, 

4.3.4, 

4.4.4, 

4.5.4, 

4.6.2,  4.6.4, 

4.7.2,  4.7.4, 

4.3.2,  4.8.4, 

4.9.2,  4.9.4, 

4.10.2,  4.10.4] 


Tools: 

[Data  Model/Dictionary 
Process  Model 
Static  Analyzers 


5.9.4, 

5.9.5, 
5.14.1] 


E&V  Reference  Manual,  Version  2.0 


7.3.1.14  Reusability  Analysis 
Description: 

The  quantitative  measurement  of  specified  reusability  factors  for  use  during  the  evaluation  of 
software  products  (and  prediction  of  software  reusability). 

Cross  References: 


Life  Cycle  Activities: 

[Preliminary  Design  4.4.2, 

Detailed  Design  4.5.2, 

Coding  and  CSU  Testing  4.6.2] 


Tools: 

[Process  Model  5.9.5, 

Design  Library  Manager  5.9.8, 

Reusable  Components  Library  5.11.8, 

Static  Analyzers  5.14.1] 


Guidebook  References: 
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7.3.1.15  Syntax  and  Semantics  Checking 


i 

Description:  1 

The  detection  of  errors  in  the  syntax  and  semantics  in  the  use  of  a  formal  language.  I 


Cross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing  4.6.2] 


Tools: 

[Word  Processor  5.8.2, 

Syntax-Directed  (Language-Sensitive)  Editor  5.11.5, 

Translating  Editor  5.12.1, 

Compiler  (Front  End)  5.12.2, 

Static  Analyzers  5.14.1] 


Guidebook  References: 

[Completeness  6.4.9, 

@GB:  Language-Sensitive  Editing  Capabilities  Checklist  99.2; 
Power  6.4.21, 

@GB:  Language-Sensitive  Editing  Capabilities  Checklist  99.2] 
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7.3.1.16  Reachability  Analysis 
Description: 

The  detection  of  sections  of  code  that  cannot  be  executed  because  of  the  structure  of  the 
code  unit  that  contains  it. 

Cross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing 

4.6.2, 

CSC  Integration  and  Testing 

4.7.2, 

CSCI  Testing 

4.8.2, 

System  Integration  and  Testing 

4.9.2, 

Operational  Testing  and  Evaluation 

4.10.2] 

Tools: 

[Static  Analyzers 


5.14.1] 
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7.3.1.17  Cross  Reference 


Description: 

The  referencing  of  entities  to  other  entities  by  logical  means.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 

[System  Requirements  Analysis/Design 
Software  Requirements  Analysis 
Preliminary  Design 
Detailed  Design 
Coding  and  CSU  Testing 
CSC  Integration  and  Testing 
CSCI  Testing 

System  Integration  and  Testing 
Operational  Testing  and  Evaluation 


4.2.3, 

4.3.3,  4.3.4, 

4.4.4, 

4.5.2  -  4.5.4, 

4.6.2,  4.6.4, 

4.7.2  -  4.7.4, 

4.8.2,  4.8.4, 

4.9.2,  4.9.4, 

4.10.2,  4.10.4] 


Tools: 

[Data  Model/Dictionary  5.9.4, 

Process  Model  5.9.5, 

Static  Analyzers  5.14.1, 

Test  Analysis  Sen/ices  5.14.6] 
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7.3.1.18  Maintainability  Analysis 
Description; 

The  quantitative  measurement  of  specified  maintainability  factors  for  use  during  the 
evaluation  of  software  products  (and  prediction  of  software  maintainability). 

Cross  References; 


Life  Cycle  Activities: 

[System  Requirements  Analysis/Design 
Software  Requirements  Analysis 
Preliminary  Design 
Detailed  Design 
Coding  and  CSU  Testing 
CSC  Integration  and  Testing 
CSCI  Testing 

System  Integration  and  Testing 
Operational  Testing  and  Evaluation 


4.2.2, 

4.3.2, 

4.4.2, 

4.5.2, 

4.6.2,  4.6.4, 

4.7.2,  4.7.4, 

4.8.2,  4.8.4, 

4.9.2,  4.9.4, 

4.10.2,  4-10.4] 


Tools: 

[Static  Analyzers 


5.14.1] 


E&V  Reference  Manual,  Version  2.0 


7.3.1.19  Invocation  Analysis 


Description: 

The  analysis  of  a  system/software  design  for  the  purpose  of  determining  calling  relationships 
between  elements.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 

[Preliminary  Design  4.4.2, 

Detailed  Design  4.5.2] 


Tools: 

[Process  Model  5.9.5. 

Applications  (Code)  Generator  5.11.1, 

Static  Analyzers  5.14.1] 
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7.3.1.20  Scanning 
Description: 

The  examination  of  entities  sequentially  to  identify  key  areas  or  structure.  [@Kean  1985] 

Cross  References: 


Life  Cycle  Activities: 


[Preliminary  Design 

4.4.4, 

Detailed  Design 

4.5.4, 

Coding  and  CSU  Testing 

4.6.4, 

CSC  Integration  and  Testing 

4.7.4] 

Tools: 

[Static  Analyzers 

5.14.1] 

Guidebook.  References: 
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7.3.1.21  Structured  Walkthrough 


Description: 

The  interactive  display  of  source  code  (or  its  design)  with  the  capability  for  branching  to 
subordinate  program  modules/units.  This  feature  automates  code  reading  such  that  a 
walkthrough  of  a  computer  program  can  be  performed  on  a  video  terminal.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 

[Preliminary  Design  4.4.2. 

Detailed  Design  4.5.2, 

Coding  and  CSU  Testing  4.6.2] 


Tools: 

[Static  Analyzers  5.14.1] 


Guidebook  References: 
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7.3.1.22  Auditing 


Description: 

The  conducting  of  an  examination  to  determine  whether  or  not  predefined  rules  have  been 
followed.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 

[System  Requirements  Analysis/Design 
Software  Requirements  Analysis 
Preliminary  Design 
Detailed  Design 
Coding  and  CSU  Testing 
CSC  Integration  and  Testing 
CSCI  Testing 

System  Integration  and  Testing 
Operational  Testing  and  Evaluation 


4.2.4, 

4.3.4, 

4.4.2,  4.4.4. 

4.5.2,  4.5.4, 

4.6.2,  4.6.4, 
4.7.2  -  4.7.4, 

4.8.2,  4.8.4, 

4.9.2,  4.9.4, 

4.10.2,  4.10.4] 


Tools: 

[Static  Analyzers 


5.14.1] 
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i 


7.3.1.23  Error  Checking 


Description: 

The  determination  of  discrepancies,  their  importance,  and/or  their  cause.  [@Kean  19851 

Cross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing 

4.6.2, 

CSC  Integration  and  Testing 

4.7.2, 

CSCI  Testing 

4.8.2, 

System  Integration  and  Testing 

4.9.2, 

Operational  Testing  and  Evaluation 

4.10.2] 

Tools: 

[Decision  Support  Services 

5.14.7] 
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7.3.1.24  Statistical  Analysis 


Description: 

The  performance  of  statistical  data  collection  and  analysis.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing  4.6.2, 

CSC  Integration  and  Testing  4.7.2, 

CSCI  Testing  4.8.2,  4.8.3, 

System  Integration  and  Testing  4.9.2, 

Operational  Testing  and  Evaluation  4  10.2] 


Tools: 

[Test  Analysis  Services  5.14.6] 


Guidebook  References: 
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7.3.1.25  Statistical  Profiling 


Description: 

The  analysis  of  a  computer  program  to  determine  statement  types,  number  of  occurrences 
of  each  statement  type,  and  the  percentage  of  each  statement  type  in  relation  to  the 
complete  program.  [@Kean  1985] 


Dross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing  4.6.2, 

System  Integration  and  Testing  4.9.3, 

Operational  Testing  and  Evaluation  4.10.3] 


Tools: 

[Test  Analysis  Services 
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7.3.1.26  Structure  Checking 


Description: 

The  detection  of  structural  flaws  within  a  program  (e.g.,  improper  loop  nestings, 
unreferenced  labels,  unreachable  statements,  and  statements  with  no  successors).  Flaws 
detected  by  this  function  are  not  illegal  or  erroneous,  but  do  constitute  bad  style.  [@Kean 
1985] 

Cross  References: 

Life  Cycle  Activities: 

[Coding  and  CSU  Testing  4.6.2] 


Tools: 

[Static  Analyzers 
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7.3.1.27  Type  Analysis 


Description: 

The  evaluation  of  whether  or  not  the  domain  of  values  attributed  to  an  entity  are  properly  and 
consistently  defined.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 


[Preliminary  Design 

4.4.2, 

Detailed  Design 

4.5.2, 

Coding  and  CSU  Testing 

4.6.2, 

CSC  Integration  and  Testing 

4.7.2, 

CSCI  Testing 

4.8.2, 

System  Integration  and  Testing 

4.9.2, 

Operational  Testing  and  Evaluation 

4.10.2] 

Tools: 

[Data  Model/Dictionary 

5.9.4. 

Static  Analyzers 
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7.3.1.28  Units 


Analysis 


Description: 

The  determination  of  whether  or  not  the  units  or  physical  dimensions  attributed  to  an  entity 
are  properly  defined  and  consistently  used.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 


[Preliminary  Design 

4.4.2, 

Detailed  Design 

4.5.2, 

Ceding  and  CSU  Testing 

4.6.2, 

CSC  Integration  and  Testing 

4.7.2, 

CSCI  Testing 

4.8.2, 

System  Integration  and  Testing 

4.9.2, 

Operational  Testing  and  Evaluation 

4.10.2] 

Tools: 

[Static  Analyzers 

5.14.1] 

Guidebook. References; 
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7.3.1.29  I/O  Specification  Analysis 


Description: 

The  analysis  of  the  input  and  output  specifications  in  a  program  usually  for  the  generation  of 
test  data.  [@Kean  1985] 

Cross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing 

4.6.2, 

CSC  Integration  and  Testing 

4.7.2, 

CSCI  Testing 

4.8.2, 

System  Integration  and  Testing 

4.9.2, 

Operational  Testing  and  Evaluation 

4.10.2] 

Tools: 

[Test  Building  Services 


5.14.3] 
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7.3.1.30  Sizing  Analysis 


Description: 

The  quantitative  measurement  of  the  maximum  amount  of  primary  (and  secondary)  storage 
required  to  run  a  program. 


Cross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing 
CSC  Integration  and  Testing 
CSCI  Testing 

System  Integration  and  Testing 
Operational  Testing  and  Evaluation 


4.6.2,  4.6.4, 

4.7.2,  4.7.4, 

4.8.2  -  4.8.4, 

4.9.2  -  4.9.4, 

4.10.2  -  4.10.4] 


Tools: 

[Size  Estimator  5.5.2, 

Test  Analysis  Services  5.14.6] 
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7.3.1.31  Data  Reduction  and  Analysis 


Description; 

The  extraction  and  processing  of  key  information  from  test  execution  logs  to  determine 
correctness  or  appropriateness  of  system  behavior  during  test  execution. 


Cross  References; 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing 
CSC  Integration  and  Testing 
CSCI  Testing 

System  Integration  and  Testing 
Operational  Testing  and  Evaluation 


4.6.2, 

4.7.2,  4.7.3, 

4.8.2,  4.8.3, 

4.9.2,  4.9.3, 

4.10.2,  4.10.3] 


Tools: 

[Test  Analysis  Services  5.14.6] 

Guidebook  References: 
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7.3.1.32  Random  Test  Generation 


Description: 

The  construction  of  test  data  sets  by  randomly  choosing  a  subset  of  all  possible  input  values. 
The  distribution  may  be  arbitrary,  or  it  may  attempt  to  accurately  reflect  the  distribution  of 
inputs  in  the  application  environment.  [@DeMillo  1987] 

Cross  References: 


Life  Cycle  Activities: 

[CSC  Integration  and  Testing 
CSCI  Testing 

System  Integration  and  Testing 
Operational  Testing  and  Evaluation 


4.7.2,  4.7.3, 

4.8.2,  4.8.3, 

4.9.2, 
4.10.2] 


Tools: 

[Test  Building  Services 
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7.3.2  Dynamic  Analysis 


Description: 

Dynamic  analysis  features  specify  operations  that  are  determined  during  or  after  execution 
takes  place.  Dynamic  analysis  features  differ  from  those  classified  as  static  by  virtue  of  the 
fact  that  they  require  some  form  of  symbolic  or  machine  execution.  They  describe  the 
techniques  used  by  the  tool  to  derive  meaningful  information  about  a  program’s  execution 
behavior.  [@Kean  1985] 

Guidebook  References: 

[Power  6.4.21, 

@GB:  SEI  Unit  Testing  and  Debugging  Experiment  7.2: 

Processing  Effectiveness  6.4.22, 

@GB:  SEI  Unit  Testing  and  Debugging  Experiment  7.2] 


7.3.2. 1  Requirements  Simulation 


Description: 

The  execution  of  code  enhanced  requirements  statements  to  examine  functional  interfaces 
and  performance.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 


[System  Concepts 

4.1.2, 

System  Requirements  Analysis/Design 

4.2.2, 

Software  Requirements  Analysis 

4.3.2] 

Tools: 

[Specifications  Language  Processor 

5.9.2, 

Simulators 

5.9.6] 

Guidebook  References: 
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7.3.2.2  Requirements  Prototyping 


Description: 

The  rapid  construction  of  critical  functions  of  a  system  early  in  the  life  cycle  for  the  purpose  of 
understanding  the  requirements.  During  this  activity,  these  results  are  intended  to  be  thrown 
away.  [@Kean  1985] 

Cross  References: 


Life  Cycle  Activities: 

[System  Concepts  4.1.2, 

System  Requirements  Ar.alysis/Design  4.2.2, 

Software  Requirements  Analysis  4.3.2] 


Tools: 

[Prototyping  Tools  5.9.7] 


Guidebook  References: 

[Completeness  6.4.9, 

@GB:  Requirements  Prototyping  Capabilities  Checklist  9.2] 
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7.3.2.3  Simulation  and  Modeling 


Description: 

The  representation  of  selected  characteristics  of  the  behavior  of  one  physical  or  abstract 
system  by  another  system  (e.g.,  the  representation  of  physical  phenomena  by  means  of 
operations  performed  by  a  computer  system,  the  representation  of  operations  of  a  computer 
system  by  those  of  another  computer  system,  etc.).  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 

[System  Concepts 

4.1.2, 

Preliminary  Design 

4.4.2] 

Tools: 

[Enterprise  Model 

5.9.3, 

Data  Model/Dictionary 

5.9.4, 

Process  Model 

5.9.5, 

Simulators 

5.9.6, 

PrototyDinq  Tools 

5.9.7, 

Host-Based  Target  System 

Instruction-Level  Simulators 

5.13.4 

Guidebook  References; 

[Completeness  6.4.9, 

@GB:  Instruction-Level  Simulation  Checklist  6.8: 

Completeness  6.4.9, 

@GB:  Simulation  and  Modeling  Capabilities  Checklist  9.3: 

Completeness  6.4.9, 

@GB:  Cross  Development  System  Support  Questionnaire  13.3] 
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7.3.2.4  Design  Prototyping 


Description: 

The  rapid  construction  of  critical  functions  of  a  system  early  in  the  life  cycle  for  the  purpose  of 
understanding  the  requirements.  During  this  activity,  the  results  will  be  retained  for  the 
purpose  of  incrementally  developing  the  product.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 

[Preliminary  Design  4.4.2, 

Detailed  Design  4.5.2] 


Tools: 

[Prototyping  Tools  5.9.7] 


Guideb<iCLK.Befgj:encfis: 
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7.3.2.5  Debugging 


Description: 

The  process  of  locating,  analyzing,  and  correcting  suspected  faults  in  a  program.  [@Kean 
1985] 


Cross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing 

4.6.2, 

CSC  Integration  and  Testing 

4.7.2, 

CSCI  Testing 

4.8.2, 

System  Integration  and  Testing 

4.9.2, 

Operational  Testing  and  Evaluation 

4.10.2] 

Tools: 

[Host-Based  Target  Code  Symbolic  Debugger  5.13.6] 


Guidebook  .References: 


[Completeness 

6.4.9, 

@GB:  Debugging  Capabilities  Checklist 

6.5; 

Completeness 

6.4.9, 

@GB:  Cross  Development  System  Support  Questionnaire 

13.3; 

Power 

6.4.21, 

@GB:  Debugging  Capabilities  Checklist 

6.5; 

Power 

6.4.21, 

@GB:  SEI  Unit  Testing  and  Debugging  Experiment 

7.2; 

Processing  Effectiveness 

6.4.22. 

@GB:  SEI  Unit  Testing  and  Debugging  Experiment 

7.2] 
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7.3.2.6  Executable  Assertion  Checking 


Description: 

The  checking  of  user-embedded  statements  that  assert  relationships  between  elements  of  a 
program.  An  assertion  is  a  logical  expression  that  specifies  a  condition  or  relation  among  the 
program  variables.  Checking  may  be  performed  with  symbolic  or  run-time  data.  [@Kean 
1985] 

Cross  References: 


Life  Cycle  Activities: 


[Coding  and  CSU  Testing 

4.6.2, 

CSC  Integration  and  Testing 

4.7.2,  4.7.3, 

CSCI  Testing 

4.8.2,  4.8.3. 

System  Integration  and  Testing 

4.9.2, 

Operational  Testing  and  Evaluation 

4.10.2] 

Tools: 

[Test  Building  Services 


5.14.3] 
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7.3.2.7  Constraint  Evaluation  (Contention) 

DgSg/ipjjon; 

The  generation  and/or  solution  of  path  input  or  output  constraints  for  determining  test  input 
proving  programs  correct.  [@Kean  1985] 

Cross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing  4  6.2] 


Tools: 

[Test  Building  Services  5.14.3] 
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7.3.2.8  Coverage/Frequency  Analysis 


Description: 

The  determination  and  assessment  of  measures  associated  with  the  invocation  of  program 
structural  elements  to  determine  the  adequacy  of  a  test  run.  Coverage  analysis  is  useful 
when  attempting  to  execute  each  statement,  branch,  path,  or  program.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 


[Coding  and  CSU  Testing 

4.6.2, 

CSC  Integration  and  Testing 

4.7.2, 

CSCI  Testing 

4.8.2, 

System  Integration  and  Testing 

4.9.2, 

Operational  Testing  and  Evaluation 

4.10.2] 

Tools: 

[Test  Building  Services 

5.14.3] 
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7.3.2.9  Mutation  Analysis 
Description; 

The  application  of  test  data  to  a  program  and  its  '‘mutants"  (i.e.,  programs  that  contain  one 
or  more  likely  errors)  in  order  to  determine  test  data  adequacy.  [@Kean  1985] 

Cross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing 

4.6.2, 

CSC  Integration  and  Testing 

4.7.2, 

CSCI  Testing 

4.8.2, 

System  Integration  and  Testing 

4.9.2, 

Operational  Testing  and  Evaluation 

4.10.2] 

Tools: 

[Test  Building  Services 


5.14.3] 
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7.3.2.10  Symbolic  Execution 
Description: 

The  reconstruction  of  the  logic  and  computations  along  a  program  path  by  executing  the  path 
with  symbolic  rather  than  actual  values  of  data.  [@Kean  1985] 

Cross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing 


4.6.2] 


Tools: 

[Test  Building  Services 
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7.3.2.11  Regression  Testing 


Description: 

The  rerunning  of  test  cases  which  a  program  has  previously  executed  correctly  in  order  to 
detect  errors  spawned  by  changes  or  corrections  made  during  software  development  and 
maintenance.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 


[Coding  and  CSU  Testing 

4.6.2, 

CSC  Integration  and  Testing 

4.7.2, 

CSCI  Testing 

4.8.2, 

System  Integration  and  Testing 

4.9.2, 

Operational  Testing  and  Evaluation 

4.10.2] 

Tools: 

[Test  Execution  Services 

5.14.5] 

Guidebook  References: 
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7.3.2.12  Resource  Utilization 


Description: 

The  analysis  of  resource  utilization  associated  with  system  hardware  or  software.  [@Kean 
1985] 


Cross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing 
CSC  Integration  and  Testing 
CSCI  Testing 

System  Integration  and  Testing 
Operational  Testing  and  Evaluation 


4.6.4, 

4.7.2,  4.7.4, 

4.8.2  -  4.8.4, 

4.9.2  -  4.9.4, 

4.10.2  -  4.10.4] 


Tools: 

[Size  Estimator  5.5.2, 

Test  Analysis  Services  5.14.6] 


Guidebook  References: 
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7.3.2.13  Emulation 


Description: 

The  imitation  of  all  or  part  of  one  computer  system  by  another,  primarily  by  hardware,  so  that 
the  imitating  computer  system  accepts  the  same  data,  executes  the  same  programs,  and 
achieves  the  same  results  as  the  imitated  system.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing  4.6.2, 

CSC  Integration  and  Testing  4.7.2, 

CSCI  Testing  4.8.2, 

System  Integration  and  Testing  4.9.2, 

Operational  Testing  and  Evaluation  4.10.2] 


Tools: 

[RAM  Cache  5.3.5, 

Prototyping  Tools  5.9.7, 

Host-Based  Target  System 

Instruction-Level  Emulator  5.13.5, 

Test  Execution  Services  5.14.5] 


Guidebook  References: 

[Completeness  6.4.9, 

@GB:  Cross  Development  System  Support  Questionnaire  13.3: 
Power  6.4.21 , 

@GB:  Emulation  Capabilities  Checklist  6.4] 
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7.3.2.14  Timing  Analysis 


Description: 

The  reporting  of  actual  CPU,  wall-clock,  or  other  times  associated  with  parts  of  a  program. 
[@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing 
CSC  Integration  and  Testing 
CSCI  Testing 

System  Integration  and  Testing 
Operational  Testing  and  Evaluation 


4.6.4, 

4.7.2,  4.7.4, 

4.8.2  -  4.8.4, 

4.9.2  -  4.9.4, 

4.10.2  -  4.10.4] 


Tools: 

[Test  Analysis  Services 


5.14.6] 


Guidebook  References: 

[Completeness  6.4.9, 

@GB:  Timing  Analysis  Capabilities  Checklist  6.6; 

Completeness  6.4.9, 

@GB:  Cross  Development  System  Support  Questionnaire  13.3] 
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7.3.2.15  Tuning 


Description: 

The  activity  of  optimizing  parts  of  a  program  which  account  for  significant  amounts  of 
resources.  The  optimization  follows  the  determination  of  a  “performance  profile”  which  is 
the  identification  of  the  parts  of  a  program  which  use  the  most  resources. 


Cross  References: 


Life  Cycle  Activities: 


[Coding  and  CSU  Testing 

4.6.2, 

CSC  Integration  and  Testing 

4.7.2. 

CSCI  Testing 

4.8.2, 

System  Integration  and  Testing 

4.9.2, 

Operational  Testing  and  Evaluation 

4.10.2] 

Tools: 

[Test  Analysis  Services 

5.14.6] 

E&V  Reference  Manual,  Version  2.0 


7.3.2.16  Reliability  Analysis 


Description: 

The  determination  of  the  ability  of  an  item  to  perform  a  required  function  under  stated 
conditions  for  a  stated  period  of  time.  [@Kean  1985] 


Cross  References: 


Life  Cycle  Activities: 


[Coding  and  CSU  Testing 

4.6.2, 

CSC  Integration  and  Testing 

4.7.2, 

CSCI  Testing 

4.8.2, 

System  Integration  and  Testing 

4.9.2, 

Operational  Testing  and  Evaluation 

4.10.2] 

Tools: 

[Test  Analysis  Services 
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7.3.2.17  Real-Time  Analysis 


Bfisscipiiapi 

The  determination  of  the  behavior  of  a  program  which  must  recognize,  interpret,  and  respond 
to  externai,  (usually)  asynchronous  events  at  speeds  which  allow  the  system  to  handle  each 
event  in  a  specified  amount  of  time. 

Cross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing  4.6.2, 

CSC  Integration  and  Testing  4.7.2,  4.7.3, 

CSCI  Testing  4.8.2, 

System  Integration  and  Testing  4.9.2, 

Operational  Testing  and  Evaluation  4.10.2] 


Tools: 

[Test  Analysis  Services  5.14.6] 


Guicte&QQ-k  References; 


[Completeness  6.4.9, 

@GB:  Real-Time  Analysis  Checklist 


6.7] 
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7.3.2.18  Path  and  Domain  Selection 


Description: 

The  construction  of  test  data  sets  by  partitioning  the  program  input  space  into  path  domains 
(those  subsets  of  the  input  space  that  cause  execution  of  each  path)  and  selecting  data 
points  from  the  domains  that  correspond  to  the  paths  of  interest.  Test  data  points  are  also 
selected  on  and  near  the  boundaries  of  the  domains  to  reveal  path  selection  errors  that  may 
exist  in  the  program  under  test.  [@DeMillo  1987] 


Cross  References: 


Life  Cycle  Activities: 

[Coding  and  CSU  Testing  4.6.2] 


Tools: 

[Test  Building  Services  5.14.3] 
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7.3.4  Problem  Report  Analysis 


Description; 

The  analysis  of  problem  reports  for  the  purpose  of  determining  the  validity  of  the  reported 
problem  and  a  corrective  action.  [@Kean  1985] 


|  Cross  References: 

ft 

^  Life  Cycle  Activities: 

ft 
I 
1 


[CSC  Integration  and  Testing 

4.7.2, 

CSCI  Testing 

4.8.2, 

System  Integration  and  Testing 

4.9.2, 

Operational  Testing  and  Evaluation 

4.10.2] 

Tools: 

[Problem  Report  Analyzer 

5.5.7, 

Decision  Support  Services 

5.14.7] 
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7.3.5  Change  Request  Analysis 
Description: 

The  analysis  of  change  requests  to  determine  the  necessity  of  the  change, 
technical/economic  impacts,  and  approach  to  accomplishing  the  change.  [@Kean  1985] 


7.3.5.1  Change  Impact  Analysis 


Description: 

The  ability  to  determine,  for  a  proposed . support/enhancement  operation,  the  impact  of 
proposed  changes  to  the  software  system.  For  example,  changes  could  be  specified  at  the 
requirements,  design,  or  code  levels  and,  utilizing  the  traceability  mechanisms  which  link 
elements  of  various  life  cycle  activities,  the  impact(s)  of  the  changes  could  be  identified. 
Impact(s)  would  include  those  to  requirements,  design,  code,  test  cases/test  data,  and 
associated  documentation.  [@Kean  1585] 

Cross  References: 

Life  Cycle  Activities: 

[Change  Requirements 

Tools: 

[Change  Impact  Analyzer 
Decision  Support  Services 


4.11.2] 

5.5.8, 

5.14.7] 
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APPENDIX  B 
GLOSSARY 


8.1  ACRONYMS  AND  ABBREVIATIONS 

This  section  provides  a  list  of  all  acronyms  and  abbreviations  used  in  this  manual, 
with  the  associated  meanings. 


ANSI 

American  National  Standards  Institute 

APSE 

Ada  Programming  Support  Environment 

ARTEWG 

Ada  Runtime  Environments  Working  Group  (SIGAda) 

CAIS 

Common  APSE  Interface  Set 

cen 

Commerce  Business  Daily 

CIDS 

Critical  Item  Development  Specification 

CRISD 

Computer  Resouices  Integrated  Support  Document 

CSC 

Computer  Software  Component 

CSCI 

Computer  Software  Configuration  Item 

CSOM 

Computer  System  Operator’s  Manual 

CSU 

Computer  Software  Unit 

DACS 

Data  and  Analysis  Center  for  Software 

DoD 

Department  of  Defense 

ECP 

Engineering  Change  Proposal 

E&V 

Evaluation  and  Validation 

FSM 

Firmware  Support  Manual 

GB 

E&V  Guidebook 

HWCI 

Hardware  Configuration  Item 

IDD 

Interface  Design  Document 

IEEE 

Institute  of  Electrical  and  Electronics  Engineers 

IPSE 

Integrated  Project  Support  Environment 

IRS 

Interface  Requirements  Specification 

MCCS 

Mission-Critical  Computer  System 

PIDS 

Prime  Item  Development  Specification 

PIWG 

Performance  Issues  Working  Group  (SIGAda) 

PSE 

Programming  Support  Environment 

RADC 

Rome  Air  Development  Center 

RFP 

Request  for  Proposal 

RM 

E&V  Reference  Manual 

SCN 

Specification  Change  Notice 

SDD 

Software  Design  Document 

SDE 

Software  Development  Environment 

SDF 

Software  Development  File 
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SDP  Software  Development  Plan 

SEE  Software  Engineering  Environment 

SIGAda  Special  Interest  Group  for  Ada  of  the  Association  for  Computing 
Machinery  (ACM) 

SOW  Statement  of  Work 

SPM  Software  Programmer’s  Manual 

SPS  Software  Product  Specification 

SRS  Software  Requirements  Specification 

SSDD  System/Segment  Design  Document 

SSS  System/Segment  Specification 

STARS  Software  Technology  for  Adaptable,  Reliable  Systems 

STD  Software  Test  Description 

STP  Software  Test  Plan 

STR  Software  Test  Report 

SUM  Software  User’s  Manual 

T&E  Test  and  Evaluation 

USDR&E  Under  Secretary  of  Defense-Research  and  Engineering 

VDD  Version  Description  Document 

WBS  Work  Breakdown  Structure 
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APPENDIX  C 
FORMAL  GRAMMAR 


This  appendix  specifies  sections  of  the  Reference  Manual  and  Guidebook  (Refer¬ 
ence  System)  as  a  formal  grammar.  The  sections  include  chapters  four  through  seven  of  the 
Reference  Manual  (RM),  all  explicit  references,  the  table  of  contents,  the  composite  index,  and 
the  reference  appendix.  The  specification  is  presented  as  a  partitioned  grammar  for  conven¬ 
ience. 

(The  grammar  is  presented  in  a  modified  Backus-Naur  form.  Brackets  represent 
optionality  when  alone,  and  may  be  marked  by  an  asterisk  " '  ”  to  denote  0-N  instances  of  the 
production,  or  by  a  sharp  to  denote  1-N  instances.  Angle  brackets  denote  comments  in 
place  of  productions  which  are  too  elaborate  to  express  here.  All  terminals  of  the  grammar  are 
expressed  as  quoted  literals,  or  composite  literals  based  on  characters  and  character  strings.) 

C.1  FORMAL  REFERENCES 

Throughout  the  Reference  System,  whenever  formal  references  are  made,  a  single 
consistent  set  of  grammar  rules  are  used.  This  includes  reference  from  one  volume  to  the  other, 
reference  from  one  section  in  a  volume  to  another  section  in  the  same  document,  and  reference 
to  documents  outside  the  Reference  System. 


referencejist 

::=  “[”  references  references  ]*  “]” 

references 

::=  reference  [  reference  J* 

reference 

::=  ["@"]  phrase  (  phrase  ]  [  designatorjist  ] 

|  [  phrase  ]  designatorjist 

phrase 

::=  <text  lacking  special  characters> 

designatorjist 

::=  designator  [  designator  ]* 
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designator  ::=  leads  |  leads  [  digits  ]* 
leads  ::=  digits  b  caps 

digits  ::=  (‘0’-’9') 

caps  ::=  (‘A*-’Z‘) 

C.2  INDEXES 

The  index  format  of  the  Reference  System  builds  on  the  reference  specifications 
above,  and  is  derived  from  MIL-STD-1815A. 

index  ::=  [  entry  ]# 

entry  ::=  topic  [  redirector  ]  (  reference Jist  ] 

topic  ::=  phrase  [  “(”  designator  Jist  “)”  ] 

redirector  ::=  “["  “see”  [  “also”  ]  phrasejist  “]” 

phrasejist  phrase  [  phrase  ]* 

C.3  FORMAL  CHAPTERS 

Those  chapters  of  the  RM  which  are  derived  from  the  classification  schema  are  for¬ 
mally  defined  here. 

C.3.1  Chapter  Components 

The  following  rules  define  the  components  which  are  used  to  compose  formal  chap¬ 
ter  entries. 

descriptor  ::=  designator  phrase  [description] 

description  ::=  “Description:”  text 

text  ::=  <  prose  text  > 
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functions  ::=  “Functions:”  referencejist 
products  ::=  “Products:”  reference-list 
gb_references  ::=  “Guidebook:”  referencejist 
life_cyc!e_activities  ::=  “Life  Cycle  Activities:"  referencejist 
tools  ::=  “Tools:”  referencejist 

qualityjactors  ::=  “Software  Quality  Factors:” 
referencejist 

acquisition_concern  ::=  “Acquisition  Concern:” 

referencejist 

software  criteria  ::=  “Software-Oriented  Criteria:” 

referencejist 

complementary Jactors  ::=  “Complementary  Software  Quality  Factors:” 

referencejist 

cooperating_criteria  ::=  “Cooperating  Criteria:” 

referencejist 

conflicting  criteria  ::=  “Conflicting  Criteria:” 

referencejist 

beneficial  Jactors  ::=  “Beneficial  Quality  Factors:” 

referencejist 

adverse  Jactors  ::=  "Adverse  Quality  Factors:” 

referencejist 

C.3.2  Chapter  Entries 

Each  numbered  section  of  the  formal  chapters  follows  a  specific  grammar  rule. 
The  following  rules  define  the  format  of  each  class  of  chapter  entries. 

Iife_cycle_activity  ::=  descriptor  management  engineering  testing  evaluation 

configuration  transition 
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management 

descriptor  products  functions 

engineering 

descriptor  products  functions 

testing 

::= 

descriptor  products  functions 

evaluation 

::= 

descriptor  products  functions 

configuration 

::= 

descriptor  products  functions 

transition 

::= 

descriptor  products  functions 

apse 

::= 

descriptor  [  toolset  ]  * 

toolset 

::= 

descriptor  [  tool  ]  * 

tool 

::= 

descriptor  functions 

attribute 

::= 

descriptor  [  concern  ]*  [factor  ] 
[  criterion  ]* 

concern 

descriptor  qualityjactors 

factor 

descriptor  acquisition_concern 

softwarecriteria 

complementaryjactors  cooperating_criteria 
conflicting_criteria 

criterion  ::=  descriptor  qualityjactors 

beneficialjactors  adversejactors 
gb_references 

function  descriptor  [  life_cycle_activities  tools  ] 

[  gbjeferences  ] 

C.3.3  Formal  Chapter  Ordering 

The  formal  portion  of  the  RM  is  found  in  chapters  four  through  seven.  Each  of  the 
classes  of  chapter  entries  is  found  in  a  distinct  chapter. 

formal_chapters  ::=  [  life_cycle_activity  ]* 

[  apse  ] 
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[  attribute  ]* 

[  function  ]* 

C.4  TABLE  OF  CONTENTS 

The  table  of  contents  shares  some  features  with  the  rest  of  the  formal  aspects  of 

the  RMGB. 

table_of_contents 
chapter 

designator_page 
index  ::= 

C.5  CITATIONS 

The  citations  are  found  in  Appendix  A,  and  have  a  formal  structure  as  defined  in  the 
following  grammar.  The  (semantic)  form  of  citation  text  is  taken  from  the  standard  for  IEEE  Soft¬ 
ware  Magazine. 


citations 

::=  [  citation  ]* 

citations 

::=  key  body  “. 

key 

::=  T  text  “]” 

body 

::=  textb  key 

::=  [  chapter  ]*  index 
=  designator  phrase  designator_page 

designator  digits 

=  “Index”  digits 
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INDEX 


Access 


[Access  Management 

7.2.37; 

Documeru  Accessibility 

6.4.13, 

System  Accessibility 

6.4.32] 

Accuracy 

6.4.1 

Activities,  Life  Cycle 

4. 

Ada  Programming  Support  Environment 
[see:  APSE] 

Adaptation 

6.3 

[Adaptation  Attributes  (of  Whole  APSEs) 

3.3.3] 

Address/Phone  Book 

5.6.3 

Analysis 

[APSE  Tool  Categories 

Analysis  and  Reporting 

5.5.9, 

Target  Code  Generation  and  Analysis  System 

5.13; 

Functions: 

7.3, 

Change  Impact  Analysis 

7.3.5. 1 , 

Change  Request  Analysis 

7.3.5, 

Coverage/Frequency  Analysis 

7. 3.2.8, 

Data  Flow  Analysis 

7.3.1. 3, 

Data  Reduction  and  Analysis 

7.3.1.31, 

Dynamic  Analysis 

7.3.2, 

Functional  Analysis 

7.3.1 .4, 

Interface  Analysis 

7.3. 1.5, 

Invocation  Analysis 

7.3.1.19, 

I/O  Specification  Analysis 

7.3.1.29, 

Maintainability  Analysis 

7.3.1.18, 

Mutation  Analysis 

7.3. 2.9, 

Problem  Report  Analysis 

7.3.4, 

Reachability  Analysis 

7.3.1.16, 

Real  Time  Analysis 

7.3.2.17, 

Reliability  Analysis 

7.3.2.16, 

Reusability  Analysis 

7.3.1.14, 

Risk  Analysis 

7. 2. 2. 9, 

Sizing  Analysis 

7.3.1.30, 

Static  Analysis 

7.3.1, 

Statistical  Analysis 

7.3.1.24, 

Test  Condition  Analysis 

7.3.1. 8. 

Testability  Analysis 

7.3.1 .7, 
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Timing  Analysis  7.3.2.14, 

Traceability  Analysis  7. 3. 1.6, 

Type  Analysis  7.3.1.27, 

Units  Analysis  7.3.1.28; 

Test  Analysis  Services  5.14.6] 


Anomaly  Management 

Application 

[Application  Independence 
Applications  (Code)  Generator 

APSE 

[see  also:  Environment] 

[APSE  Tool  Categories 
Distributed  APSE  Support 
Life  Cycle  Activities 
Whole  APSE  Assessment  Issues 
Approaches  to  Whole-APSE  E&V 
APSE  Definitions  and  Alternate  Names 
APSE  Viewed  as  a  Collection  of  Tools 
APSE  Viewed  as  a  Knowledge-Based  Expert  System 
APSE  Viewed  as  a  Methodology-Support  System 
APSE  Viewed  as  a  Stable  Framework 
APSE  Viewed  as  a  User-Oriented,  Interactive  System 
APSE  Viewed  as  an  Information  Management  System 
Kernel  APSE  (KAPSE) 

Key  Attributes  of  Whole  APSEs 
Minimal  APSE  (MAPSE) 

Views  of  an  APSE 

Archive,  Backup,  and  Retrieval  System 

Assemble 

[Assembler 

Assembling 

Host-Target  System  Cross-Assembler 

Assertion  Checking,  Executable 

Assistance 

[see:  On-Line  Assistance] 

Attributes 

[Central  Role  of  Attributes  and  Functions 
Key  Attributes  of  Whole  APSEs 

Auditing 

Augmentability 

Autonomy 


6.4.2 

6.4.3; 

5.11.1] 


5., 

5.4; 

4.; 


3.1, 

3.2.1, 

3.2.5, 

3.2.2, 

3.2.6, 
3.2.4, 

3.2.3, 
3.2.1, 

3.3, 
3.2.1, 
3.2] 

5.1.2 


5.12.4; 
7.1.6. 6; 
5.13.1] 

7.3. 2. 6 


6. 

2.3; 

3.3] 

7.3.1.22 

6.4.4 

6.4.5 
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Backup,  and  Retrieval  System,  Archive, 

Benchmarks  and  Test  Suites 

Bindings  to  Standard  Interface  Specification 
Implementations,  Language 

Body  Stub  Generation 

Calculator 

Calendar 

Capacity 

Change 

[see  also:  Product:  Engineering  Change  Proposal, 
Specification  Change  Notice] 
[Change  Impact  Analysis 
Change  Impact  Analyzer 
Change  Request  Analysis 
Change  Requirements 

Checklists  and  Questionnaires 

CIDS 

[see:  Product:  Critical  Item  Development  Specification] 

Classification  Schema 
Code 

[Applications  (Code)  Generator 
Coding  and  CSU  Testing 
Compiler  (Code  Generator  —  Back  End) 

Target  Code  Generation  and  Analysis  System 

Command  Language 

[Command  Language  Processing 
Command  Language  Interface 

Commonality  (Data  and  Communication) 

Communicate 

[Commonality  (Data  and  Communication) 
Communication  Effectiveness 
Operability  (Communicativeness) 

Software  and  System  Test  Communications 

Comparison 

Compile 

[Compilation 
Compilation  System: 


5.1.2 

3.4.1 


5.3.3 

7.1.6.11 

5.6.2 

5.6.6 

6.4.6 


7.3.5.1: 

5.5.8; 

7.3.5: 

4.11] 

3.4.2 


2.1 


5.11.1; 

4.6; 

5.12.3; 

5.13] 


7.2.3. 1; 
5.1.1] 

6.4.7 


6.4.7, 

6.4.8, 
6.4.20; 
7.1.6.15] 

7.3.1. 1 


7. 1.6. 7; 
5.12, 
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Compiler  (Code  Generator  —  Back  End)  5.12.3 

Compiler  (Front  End)  5.12.2] 

Completeness  6.4.9 

[Completeness  Checking  7.3.1.12] 

Complexity  Measurement  7.3.1.10 

Component 

[APSE  Tool  Categories  5., 

Reusable  Components  Library  5.11.8, 

Tool  Support  Components  5.3] 

Compression  7.1.6.16 

Computer 

[see  also:  Product:  Computer  Resources  Integrated  Support 
Document] 

[Computer  Management  System  5.1: 

Computer  System  Management  7.2.3] 

Concepts,  System  4.1 

Conferencing 

[see:  Electronic] 

Configuration 

[Configuration  Management  7.2.2.7; 

Life  Cycle  Activities:  Configuration  Management  4.x.5; 

Configuration  Management  System  5.7, 

Configuration  Control  5.7.2, 

Configuration  Identification  5.7.1, 

Configuration  Status  Accounting  5.7.3: 

Required  Configuration  6.4.26] 

Consistency  6.4.10 

[Consistency  Checking  7.3.1.13] 

Constraint  Evaluation  7.3.2.7 

Contention 

[see:  Constraint] 

Conversion  7.1. 6.8 

Correctness  6.2.1 

[Correctness  Checking  7.3.1.11] 

Cost  6.4.11 

[Cost  Estimator  5.5.1; 

Cost  Estimation  7.2.2. 1] 

Coverage/Frequency  Analysis  7.3.2.8 
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CRISD 

[see:  Product:  Computer  Resources  Integrated  Support  Document] 


Cross  Reference  7.3.1 . . . 

CSC  Integration  and  Testing  4.7 

CSCI  Testing  4.8 

CSOM 

[see:  Product:  Computer  System  Operator’s  Manual] 

CSU  Testing,  Coding,  and  4.6 

Data 

[Data  and  Error  Logging  7.2.1.11; 

Data  Definition  Language  Processor  5.11.6; 

Data  Editing  7. 1.1. 2, 

Data  Flow  Analysis  7.3. 1.3; 

Data  Manipulation  Language  Processor  5.11.7, 

Data  Model/Dictionary  5.9.4; 

Data  Reduction  and  Analysis  7.3.1.31, 

Database  Management  7.2. 1.1; 

Database  Manager  5.2.2. 

Database  Schema  Generator  5.11.2; 

Test  Data  Management  7.2. 1.8] 

Debug 

[Debugging  7. 3. 2. 5; 

Host-Based  Target  Code  Symbolic  Debugger  5.13.6] 

Decision 

[Decision  Aids  3.4.4; 

Decision  Support  Services  5.14.7] 

Decompilation  7.1. 7.5 

Decryption  7.1.6.19 

Definition  Language  Processor  5.9.1 

Design  6.2 

[see  also:  Product:  Interface  Design  Document, 

Software  Design  Document, 

System/Segment  Design  Document] 

[Design  Attributes  (of  Whole  APSEs)  3.3.2; 

Design  Generation  7. 1.7.1; 

Design  Library  Manager  5.9.8; 

Design  Prototyping  7. 3. 2. 4; 

Detailed  Design  4.5; 

Detailed  Design  Translation  7. 1.6. 5; 

Preliminary  Design  4.4; 
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Preliminary  Design  Translation 
Requirements/Desian  Support 
System  Requirements  Analysis/Design 

Desktop  System 

Detailed  Design 

[see:  Design] 

Development 

[see  also:  Methodology] 

[Distributed  System  Development  and  Runtime  Support 
Life  Cycle  Activities:  System  Development  Management 

Dictionary 

[Data  Model/Dictionary 
Dictionary/Thesaurus 

Disassembling 

Display,  Status 

Distributed 

[Distributed  APSE  Support 

Distributed  System  Development  and  Runtime  Support 
Distributedness 

Document 

[see  also:  Product] 

[Document  Accessibility 
Document  Generation  System 
Document  Manager 
Documentation  Management 

Domain  Selection,  Path  and 

Downloaded  Host-to-Target 

Dynamic  Analysis 

E&V 

[Approaches  to  Whole-APSE  E&V 
The  Need  for  E&V  Technology 

ECP 

[see:  Product:  Engineering  Change  Proposal] 

Edit 

[Editing: 

Data  Editing 
Graphics  Editing 
Text  Editing 

Syntax-Directed  (Language-Sensitive)  Editor 


7. 1.6. 4; 
5.9: 

4.2] 

5.6 


5.10; 

4.X.1] 


5.9.4, 

5.6.7] 

7.1. 7.6 

7.2.1.12 


5.4, 

5.10; 

6.4.12] 


6.4.13; 
5.8, 
5.8.1; 
7.2. 1.2] 

7.3.2.18 

5.13.7 

7.3.2 


3.4; 

1.3] 


7.1.1, 

7.1. 1.2, 

7.1. 1.3, 
7.1. 1.1; 
5.11.5, 
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Translating  Editor 
Word  Processor 

Efficiency 

Electronic 

[APSE  Tool  Categories: 

Electronic  Conferencing 
Electronic  Mail 
Functions: 

Electronic  Conferencing 
Electronic  Mail 

Emulate 

[Emulation 

Host-Based  Target  System  Instruction-Level  Emulator 
Encryption 
Engineering,  System 
Enterprise  Model 
Environment 

[Ada  Programming  Support  Environment  (APSE) 
Framing  Environment 
General  Environment 

Integrated  Project  Support  Environment  (IPSE) 
Language-Centered  Environment 
Method-Based  Environment 
Programming  Environment 
Programming  Support  Environment  (PSE) 

Software  Development  Environment  (SDE) 

Software  Engineering  Environment  (SEE) 
Structure-Oriented  Environment 
Toolkit  Environment 

Error 

[Data  and  Error  Logging 
Error  Checking 

Error  Tolerance  (Anomaly  Management) 

Estimate 

[Cost  Estimation 
Cost  Estimator 
Resource  Estimation 
Resource  Estimator 
Size  Estimator 

Evaluation 

[see  also:  E&V] 

[Constraint  Evaluation 


5.12.1, 

5.4.2] 

6.1.1 


5.6.5, 
5.6.4; 

7.2. 1.5, 
7.2.1. 4] 


7.3.2.13: 

5.13.5] 

7.1.6.18 

4.x. 2 

5.9.3 


3.1, 

3.1, 

3.1, 

3.1, 

3.1, 

3.1, 

3.1, 
3.1, 
3.1, 
3.1, 
3.1, 
3.1] 


7.2.1.11, 

7.3.1.23; 

6.4.2] 


7.2.2.1; 

5.5.1: 

7. 2. 2. 5; 

5.5.5, 

5.5.2] 


7.3 .2.7, 
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Evaluation  Results  Management 

Life  Cycle  Activities:  System  Product  Evaluations 

Operational  Testing  And  Evaluation 

Execute 

[Executable  Assertion  Checking 
Processing  (Execution)  Effectiveness 
Symbolic  Execution 
Test  Execution  Services 

Expand 

[Expandability 
Expansion 
Macro  Expansion 

Experiments,  Structured 

Expert  System,  APSE  Viewed  as  a  Knowledge-Based 

Export  System,  Import/ 

[see:  Import/Export] 

Fault  Tolerance 

File 

[File  Management 
File  Manager 

Flexibility 

Forms,  Predefined  and  User-Defined 

Formal 

[Formal  Verification 

Life  Cycle  Activities:  Formal  Qualification  Testing 

Format 

[Formatter 

Formatting: 

MIL-STD  Format 
Table  of  Contents 

Framework,  APSE  Viewed  as  a  Stable 

Frequency  Analysis,  Coverage/ 

FSM 

[see:  Product:  Firmware  Support  Manual] 

Function 

[Functional  Analysis 
Functional  Overlap 
Functional  Scope 
Functions 


7.2.1. 9: 
4.x. 4, 
4.10] 


7.3.2.61 

6.4.22; 

7.3.2.10; 

5.14.5] 


6.3.1; 
7.1.6.17, 
7.1. 6.9] 

3.4.3 

3.2.5 

5.1.6 


6.4.2 


7.2. 1.3; 
5.2.1] 

6.3.1 

7.1. 2.3 


7.3.3; 
4.x. 3] 


5.8.5; 

7.1.2, 

7. 1.2.1, 
7.1. 2.2] 

3.2.6 

7.3. 2. 8 


7. 3. 1.4; 
6.4.14, 
6.4.15; 
7-] 
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Generality 


Generate 

[Body  Stub  Generation 
Design  Generation 
Document  Generation  System 
Graphics  Generation 
Graphics  Generator 
Preamble  Generation 
Program  Generation 
Random  Test  Generation 
Target  Code  Generation  and  Analysis 
Test  Harness  Generation 


System 


Global 


Granularity 

Graphics 

[Graphics  Editing 
Graphics  Generation 
Graphics  Generator 

Harness  Generation,  Test 
Help 

[see:  On-Line  Assistance] 

History 

I/O 

[I/O  Pipes 

I/O  Specification  Analysis 
Input  and  Output  Services 
Input/Output  Support 

IOO 

[see:  Product:  Interface  Design  Document] 

Impact  Analysis,  Change 
Implementation  Support 

Import/Export 

[Import/Export  System 

Input 

[see:  I/O] 

Index 

[Index  Relationships 
Using  the  Reference  Indexes  Directly 


6.4.16 


7.1.6.11, 
7.1. 7.1; 
5.8; 
7.1.5; 
5.8.4; 

7.1.6.12, 
7.1.7. 3, 
7.3.1.32; 
5.13; 

7. 1.7.7] 

4.12 

6.4.17 


7.1. 1.3, 

7.1.5; 

5.8.4] 

7.1. 7.7 


5.7.5 


5.3.4; 
7.3.1.29; 
5.1.8; 
7.2.3. 2] 


7. 3.5.1 

5.11 

7.2.3.6 

5.1.6] 


2.4; 

2.3.1] 
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Information  Management 

[APSE  Viewed  as  an  Information  Management  System 
Functions:  Information  Management 
Information  Management  System 

Integration 

[CSC  integration  And  Testing 
System  Integration  And  Testing 

Integrity 

Interactive  System,  APSE  Viewed  as  a  User-Oriented 
Interface 

[see  also:  Product:  Interface  Design  Document, 

Interface  Requirements  Specification] 
[Command  Language  Interface 
Interface  Analysis 

Language  Bindings  to  Standard  Interface 
Specification  Implementations 
Layered  Portable  Interfaces 

Interoperability 

Interpret 

[Interpretation 

Interpreter 

Invocation  Analysis 
IPSE 

[see  also:  APSE,  Environment] 

[Life  Cycle  Activities 

IRS 

[see:  Product:  Interface  Requirements  Specification] 

Job 

[Job  Scheduler 
Job  Scheduling 

Kernel 

Language 

[see  also:  Command  Language] 

[Data  Definition  Language  Processor 
Data  Manipulation  Language  Processor 
Definition  Language  Processor 
Requirements  to  Natural  Language  Translation 
Specification  Language  Processor 
Syntax-Directed  (Language-Sensitive)  Editor 

Library 

[Design  Library  Manager 
Program  Library  Management 


3.2.3: 

7.2.1; 

5.2] 


4.7, 

4.9] 

6.1.2 

3.2.4 


5.1.1: 

7.3.1.5: 

5.3.3; 

3.2.6] 

6.3.2 


7.1.6.14; 

5.12.5] 


7.3.1.19 


4-] 


5.1.4; 

7. 2. 3. 8] 

7.2.3. 3 


5.11.6, 

5.11.7, 
5.9.1: 
7.1.6.3: 
5.9.2, 
5.11.5] 


5.9.8: 
7.2.1. 7: 
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Program  Library  Manager 
Reusable  Components  Library 

Life  Cycle  Activities 

Link 

[Host-Based  Target  Linker 
Unking/Loading 

Load 

[Host-Based  Target  Loader 
Linking/Loading 

Logging,  Data  and  Error 

Macro  Expansion 

Mail 

[see:  Electronic] 

Maintainability 

[Maintainability  Analysis 

Management 

[APSE  Tool  Categories: 

Computer  Management  System 
Configuration  Management  System 
Information  Management  System 
Project  Management  System 
APSE  Viewed  as  an  Information  Management  System 
Attributes: 

Anomaly  Management 
Functions: 

Access  Management 
Computer  System  Management 
Configuration  Management 
Database  Management 
Documentation  Management 
Evaluation  Results  Management 
File  Management 
Information  Management 
Program  Library  Management 
Project  Management 
Resource  Management 
Specification  Management 
Test  Data  Management 
Life  Cycle  Activities: 

Configuration  Management 
Software  Development  Management 

Math/Statistics 


5.12.6. 

5.11.8] 

4. 


5.13.2; 

7.1.6.13] 


5.13.3: 

7.1.6.13] 

7.2.1. 1 

7.1. 6. 9 


6.2.2 

7.3.1.18] 


5.1. 

5.7, 

5.2. 

5.5: 

3.2.3: 

6.4.2; 

7.2. 

7. 2. 3. 7, 
7.2.3. 

7.2.2 .7, 

7.2.1. 1, 

7. 2. 1.2, 

7. 2. 1.9, 

7.2.1. 3, 

7.2.1, 

7.2.1. 7, 

7.2.2, 

7. 2. 3. 9, 
7.2.1. 6, 
7.2. 1.8; 

4.x. 5, 
4.x.  1] 

7. 2. 3. 4 
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Maturity  6.4.18 

Measurement 

[Complexity  Measurement  7.3.1.10, 

Quality  Measurement  7.3. 1.9] 

Merge,  Sort/  7.1.4 

Methodology-Support  System,  APSE  Viewed  as  a  3.2.2 

Model 

[Data  Model/Dictionary  5.9.4, 

Enterprise  Model  5.9.3, 

Process  Model  5.9.5; 

Simulation  And  Modeling  7. 3. 2. 3] 

Modularity  6.4.19 

Monitor 

[Performance  Monitor  5.1.9; 

Performance  Monitoring  7.2.1.10] 

Mutation  Analysis  7.3. 2.9 

Natural  Language  Translation,  Requirements  to  7.1. 6.3 

On-Line  Assistance 

[On-Line  Assistance  Processing  7.1.3; 

On-Line  Assistance  5.1.7] 

Operability  6.4.20 

Operating  System,  Virtual  5.3.1 

Operational  Testing  and  Evaluation  4.10 

Output 

[see:  I/O] 

Path  and  Domain  Selection  7.3.2.18 

Performance  6.1 

[Performance  Attributes  (of  Whole  APSEs)  3.3.1; 

Performance  Monitor  5.1.9; 

Performance  Monitoring  7.2.1.10] 

Phone  Book,  Address/  5.6.3 

PIDS 

[see:  Product:  Prime  Item  Development  Specification] 

Power  6.4.21 

Preamble  Generation  7.1.6.12 
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Preliminary  Design 

[see:  Design] 

Problem  Report 

[Problem  Report  Analysis 
Problem  Report  Ana'yzer 

Process 

[Process  Model 

Processing  (Execution)  Effectiveness 

Product 

[Computer  Resources  Integrated  Support  Document  (CRISD): 
Detailed  Design 
Preliminary  Design 
Software  Requirements  Analysis 
System  Requirements  Analysis/Design 
Computer  System  Operator's  Manual  (CSOM): 

Coding  and  CSU  Testing 
CSC  Integration  and  Testing 
CSCI  Testing 
Detailed  Design 
Preliminary  Design 

Critical  Item  Development  Specification  (CIDS): 

System  Requirements  Analysis/Design 
Engineering  Change  Proposal  (ECP): 

Change  Requirements 
Firmware  Support  Manual  (FSM): 

Coding  and  CSU  Testing 
CSC  Integration  and  Testing 
CSCI  Testing 
Detailed  Design 

Interface  Design  Document  (IDD): 

CSCI  Testing 
Detailed  Design 
Preliminary  Design 

Interface  Requirements  Specification  (IRS): 

Software  Requirements  Analysis 
System  Requirements  Analysis/Design 
Life  Cycle  Activities:  System  Product  Evaluations 
Prime  Item  Development  Specification  (PIDS): 

System  Requirements  Analysis/Design 
Software  Design  Document  (SDD): 

CSCI  Testing 
Detailed  Design 
Preliminary  Design 


7.3.4; 

5.5.7] 


5.9.5; 

6.4.22] 


4.5.6, 

4.4.6, 

4.3.6, 
4.2.6; 

4.6.6, 

4.7.6, 

4.8.6, 

4.5.6, 
4.4.6; 

4.2.2; 

4.11.2; 

4.6.6, 

4.7.6, 

4.8.6, 
4.5.6; 

4.8.2, 

4.5.2, 
4.4.2; 

4.3.2, 
4.2.2; 
4.X.4; 

4.2.2; 

4.8.2, 

4.5.2, 
4.4.2; 
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Software  Development  File  (SDF): 

Coding  and  CSU  Testing 
CSC  Integration  and  Testing 
CSCI  Testing 
Detailed  Design 

Software  Development  Plan  (SDP): 

Coding  and  CSU  Testing 
CSC  Integration  and  Testing 
CSCI  Testing 
Detailed  Design 
Preliminary  Design 
Software  Requirements  Analysis 
System  Requirements  Analysis/Design 
Software  Product  Specification  (SPS): 

CSCI  Testing 

System  Integration  and  Testing 
Software  Programmer's  Manual  (SPM): 
Coding  and  CSU  Testing 
CSC  Integration  and  Testing 
CSCI  Testing 
Detailed  Design 

Software  Requirements  Specification  (SRS): 
Software  Requirements  Analysis 
System  Requirements  Analysis/Design 
Software  Test  Description  (STD): 

Coding  and  CSU  Testing 
CSC  Integration  and  Testing 
CSCI  Testing 
Detailed  Design 
Software  Test  Plan  (STP); 

Preliminary  Design 
Software  Test  Report  (STR): 

CSCI  Testing 

Software  User’s  Manual  (SUM): 

Coding  and  CSU  Testing 
CSC  Integration  and  Testing 
CSCI  Testing 
Detailed  Design 
Preliminary  Design 
Specification  Change  Notice  (SCN): 

Change  Requirements 

System/Segment  Design  Document  (SSDD): 

System  Requirements  Analysis/Design 
System/Segment  Specification  (SSS): 
System  Requirements  Analysis/Design 

I 


4.6.2, 

4.7.2, 

4.8.2, 
4.5.2; 

4.6.1, 

4.7.1, 

4.8.1, 

4.5.1, 

4.4.1, 

4.3.1, 
4.2.1; 

4.8.2, 
4.9.2; 

4.6.6, 

4.7.6, 

4.8.6, 
4.5.6; 

4.3.2, 
4.2.2; 

4.6.3, 

4.7.3, 

4.8.3, 
4.5.3; 

4.4.3; 

4.8.3; 

4.6.6, 

4.7.6, 

4.8.6, 

4.5.6, 
4.4.6; 

4.11.5 

4.2.2; 

4.2.2; 
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Version  Description  Document  (VDD): 


CSCI  Testing 

System  Integration  and  Testing 

4.8.5, 

4.9.5] 

Program 

[see  also:  Product:  Software  Programmer’s  Manual] 
[Program  Generation 

Program  Library  Management 

Program  Library  Manager 

7.1. 7.3, 
7.2.1 .7; 
5.12.6] 

Project  Management 

[Project  Management  System 

7.2.2 

5.5] 

Proprietary  Rights 

6.4.23 

Prototype 

[Design  Prototyping 

Prototyping  Tools 

Requirements  Prototyping 

7.3.2.4; 

5.9.7; 

7. 3. 2. 2] 

Qualification  Testing,  Formal 

4.X.3 

Quality 

[Quality  Measurement 

Quality  Analyzer 

Quality  Assessment 

Quality  Specification 

7.3.1 .9; 
5.2.2; 

7. 2. 2. 8, 
7. 2. 2. 2] 

Questionnaires,  Checklists  and 

3.4.2 

RAM  Cache 

5.3.5 

Random  Test  Generation 

7.3.1.32 

Reachability  Analysis 

7.3.1.16 

Real  Time  Analysis 

7.3.2.17 

Reconfigurability 

6.4.24 

Reduction  and  Analysis,  Data 

7.3.1.31 

Reference 

[Cross  Reference 

Structure  and  Use  of  the  Reference  Manual 

7.3.1.17; 

2.] 

Regression  Testing 

7.3.2.11 

Rehostability 

6.4.25 

Reliability 

[Reliability  Analysis 

6.1.3 

7.3.2.16] 

Report 

[see  also:  Product:  Software  Test  Report] 

[Analysis  and  Reporting 

5.5.9; 
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Change  Report  Analysis 

7.3.5, 

Problem  Report  Analysis 

7.3.4; 

Report  Generator 

5.11.3] 

Requirements 

[see  also:  Product:  Interface  Requirements  Specification, 

Software  Requirements  Specification] 

[Change  Requirements 

4.11; 

Required  Configuration 

6.4.26; 

Requirements/Design  Support 

5.9; 

Requirements  P.  ototyping 

7. 3. 2. 2. 

Requirements  Reconstruction 

7.1 .7.2, 

Requirements  Simulation 

7.3.2. 1 , 

Requirements  to  Natural  Language  Translation 

7.1. 6.3; 

Software  Requirements  Analysis 

4.3; 

Software  Requirements  Translation 

7.1.6. 1; 

System  Requirements  Analysis/Design 

4.2; 

System  Requirements  Translation 

7.1 .6.2] 

Resource 

[see  also:  Product:  Computer  Resources  Integrated  Support  Document] 

[Resource  Controller 

5.1.5; 

Resource  Estimation 

7. 2. 2. 5; 

Resource  Estimator 

5.5.5; 

Resource  Management 

7.2. 3.9, 

Resource  Utilization 

7.3.2.12] 

Retargetability 

6.4.27 

Retrieval  System,  Archive,  Backup,  and 

5.1.2 

Reusable 

[Reusable  Components  Library 

5.11.8; 

Reusability 

6.3.3; 

Reusability  Analysis 

7.3.1.14] 

Risk  Analysis 

7.2.2. 9 

Robustness 

6.4.2 

Runtime 

[Distributed  System  Development  and  Runtime  Support 

5.10; 

Runtime  Environment 

7. 2. 3. 5; 

Runtime  System 

5.12.7] 

Scanning 

7.3.1.20 

Schedule 

[Job  Scheduler 

5.1.4; 

Job  Scheduling 

7. 2. 3. 8; 

Scheduler 

5.5.3; 

Scheduling 

7.2.2.3J 
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Schema  Generator,  Database 
SCN 

[see:  Product:  Specification  Change  Notice] 

Screen  Generator 
SDD 

[see:  Product:  Software  Design  Document] 

SDF 

[see:  Product:  Software  Development  File] 

SDP 

[see:  Product:  Software  Development  Plan] 

Security  System 

Selection,  Path  and  Domain 

Self-Descriptiveness 

Simplicity 

Simulate 

[Host-Based  Target  System  Instruction-Level  Simulator 
Requirements  Simulation 
Simulation  and  Modeling 
Simulators 

Size 

[Size  Estimator 
Sizing  Analysis 

Software 

[Software  and  System  Test  Communications 
Software-Oriented  Criteria: 

Software  Production  Vehicle  (s) 

Software  Requirements  Analysis 
Software  Requirements  Translation 

Sort/Merge 

Source  Reconstruction 
Specification 

[see  also  :  Product:  Critical  Item  Development  Specification, 
Interface  Requirements  Specification, 
Prime  Item  Development  Specification, 
Software  Product  Specification, 
Software  Requirements  Specification, 
System/Segment  Specification] 

[I/O  Specification  Analysis 


5.11.2 


5.11.4 


5.1.3 

7.3.2.18 

6.4.28 

6.4.29 


5.13.4; 
7.3.2. 1 , 
7.3.2.31 
5.9.6] 


5.5.2; 

7.3.1.30] 


7.1.6.15: 

6.4, 

6.4.30; 

4.3; 

7.1. 6.2] 

7.1.4 

7.1. 7.4 


7.3.1.29; 
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Language  Bindings  to  Standard  Interface 
Specification  Implementations 
Quality  Specification 
Specification  Language  Processor 
Specification  Management 

Spelling 

[Spell  Checker 
Spelling  Checking 

SPM 

[see:  Product:  Software  Programmer’s  Manual] 

Spreadsheet 

SPS 

[see:  Product:  Software  Product  Specification] 

SRS 

[see:  Product:  Software  Requirements  Specification] 

SSDD 

[see:  Product:  System/Segment  Design  Document] 

sss 

[see:  Product:  System/Segment  Specification] 

Static 

[Static  Analysis 
Static  Analyzers 

Statistic 

[Math/Statistics 
Statistical  Analysis 
Statistical  Profiling 

Status  Display 

STD 

[see:  Product:  Software  Test  Description] 

Storage  Effectiveness 
STP 

[see:  Product:  Software  Test  Plan] 

STR 

[see:  Product:  Software  Test  Report] 

Structure 

[Object  Structure  Layer 
Structure  and  Use  of  the  Reference  Manual 
Structure  Checking 
Structure  Preprocessing 


5.3.3: 
7.2. 2. 2; 
5.9.2: 
7.2. 1.6] 


5.8.3: 
7.3.1. 2] 


5.6.1 


7.3.1: 

5.14.1] 


7. 2. 3. 4, 

7.1.3.24, 

7.1.3.25] 

7.2.1.12 


6.4.31 


3.2.6; 

2.: 

7.3.1.26, 

7.1.6.10] 
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SUM 

[see:  Product:  Software  User’s  Manual] 

Support 

[see  also:  Product:  Computer  Resources  Integrated  Support  Document] 


[APSE  Tool  Categories 

Distributed  APSE  Support  5.4, 

Distributed  System  Development  and  Runtime  Support  5.10, 

Implementation  Support  5.11, 

Requirements/Design  Support  5.9, 

Tool  Support  Components  5.3: 

Life  Cycle  Activities:  Transitioning  to  System  Support  4.x. 6] 

Survivability  6.1.4 

Symbolic  Execution  7.3.2.10 

Syntax 

[Syntax-Directed  (Language-Sensitive)  Editor  5.11.5: 

Syntax  And  Semantics  Checking  7.3.1.15] 

Synthesis  7.1.7 


System 

[see  also:  Product:  Computer  System  Operator’s  Manual, 


System/Segment  Design  Document, 

System/Segment  Specification] 

[Distributed  System  Development  and  Runtime  Support  5.10: 

Life  Cycle  Activities: 

System  Concepts  4.1, 

System  Development  Management  4.x.1, 

System  Engineering  4.x.2, 

System  Integration  and  Testing  4.9, 

System  Product  Evaluations  4.x. 4, 

System  Requirements  Analysis/Design  4.2, 

Transitioning  To  System  Support  4.x. 6; 

Software  and  System  Test  Communications  7.1.6.15: 

System  Accessibility  6.4.32, 

System  Clarity  6.4.33, 

System  Compatibility  6.4.34; 

System  Requirements  T ranslation  7 . 1 . 6 . 1  ] 

Table  of  Contents  Formatting  7.1. 2.2 

Target 

[Target  Code  Generation  and  Analysis  System  5.13, 

Host-Based  Target  Code  Symbolic  Debugger  5.13.6, 

Host-Based  Target  Linker  5.13.2, 

Host-Based  Target  Loader  5.13.3, 

Host-Based  Target  System  Instruction-Level  Emulator  5.13.5, 


index- 19 


E&V  Reference  Manual,  Version  2.0 


Host-Based  Target  System  Instruction-Level  Simulator 

5.13.4, 

Host-Target  System  Cross-Assembler 

5.13.1, 

Host-to-Target  Downloader 

5.13.7, 

Target-to-Host  Uploader 

5.13.8] 

Test 

[see  also:  Product:  Software  Test  Description, 

Software  Test  Plan, 

Software  Test  Procedure, 

Software  Test  Report] 

[Life  Cycle  Activities: 

Coding  and  CSU  Testing 

4.6, 

CSC  Integration  and  Testing 

4.7, 

CSCI  Testing 

4.8, 

Formal  Qualification  Testing 

4.x. 3, 

Operational  Testing  and  Evaluation 

4.10, 

System  Integration  and  Testing 

4.9; 

Random  Test  Generation 

7.3.1.32, 

Regression  Testing 

7.3.2.11, 

Software  and  System  Test  Communications 

7.1.6.15; 

Test  Analysis  Services 

5.14.6, 

Test  Building  Services 

5.14.3: 

Test  Condition  Analysis 

7.3.1. 8. 

Test  Data  Management 

7.2.1. 8: 

Test  Description  and  Preparation  Services 

5.14.4, 

Test  Execution  Services 

5.14.5: 

Test  Harness  Generation 

7.1. 7.7; 

Test  Suites,  Benchmarks  and 

3.4.1; 

Test  System 

5.14; 

Testability  (Verifiability) 

6.2.3: 

Testability  Analysis 

7.3.1. 7] 

Text  Editing 

7.1. 1.1 

Thesaurus,  Dictionary/ 

5.6.7 

Timing  Analysis 

7.3.2.14 

Tool 

[APSE  Tool  Categories 

5., 

Tool  Building  Services 

5.14.2, 

Tool  Support  Components 

5.3; 

APSE  Viewed  as  a  Collection  of  Tools 

3.2.1] 

Traceability  Analysis 

7.3. 1.6 

Tracking 

[Function 

7. 2. 2. 6: 

Tool 

5.5.6] 
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Training 

6.4.36 

Transitioning  to  System  Support 

4.x. 6 

Translate 

[Translating  Editor 

Translation 

5.12.1: 

7.1.6] 

Transportability 

6.3.4 

Tuning 

7.3.2.15 

Type  Analysis 

7.3.1.27 

Unit 

[Coding  and  CSU  Testing 

Units  Analysis 

4.6: 

7.3.1.28] 

Uploader,  Target-to-Host 

5.13.8 

Usability 

6.1.5 

Validation 
[see:  E&V] 

VDD 

[see:  Product:  Version  Description  Document] 

Verifiability 

6.2.3 

Verification,  Formal 

7.3.3 

Version  Control 

5.7.4 

Virtual 

[Virtual  Layer 

Virtual  Operating  System 

Virtuality 

3.2.6: 

5.3.1: 

6.4.37] 

Visibility 

6.4.38 

Walkthrough,  Structured 

7.3.1.21 

Whole  APSE 
[see:  APSE] 

Window  Manager 

5.3.2 

Word  Processor 

5.8.2 

Work  Breakdown  Structure 
[Function 

Work  Breakdown  Structure  Editor 

7.2. 2. 4; 
5.5.4] 
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